Systems and methods for performing data replication

ABSTRACT

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 12/692,450, filed Jan. 22, 2010, which is a continuation ofU.S. patent application Ser. No. 11/640,826, filed Dec. 18, 2006, nowU.S. Pat. No. 7,651,593, issued Jan. 26, 2010, which claims the benefitof priority under 35 U.S.C. §119(e) from U.S. Provisional ApplicationNo. 60/833,851, filed Jul. 27, 2006, and entitled “SYSTEMS AND METHODSFOR CONTINUOUS DATA REPLICATION,” and U.S. Provisional Application No.60/752,204, filed Dec. 19, 2005, and entitled “SYSTEM AND METHOD FORESTABLISHING REFERENTIAL CONSISTENCY IN STORAGE OPERATIONS,” theentirety of each of which is hereby incorporated herein by reference.

The present application is also related to the following applicationsfiled on even date herewith, each of which is hereby incorporated hereinby reference in its entirety:

-   -   U.S. patent application Ser. No. 11/640,825, entitled “BUFFER        CONFIGURATION FOR A DATA REPLICATION SYSTEM;”    -   U.S. patent application Ser. No. 11/640,830, entitled “PATHNAME        TRANSLATION IN A DATA REPLICATION SYSTEM;”    -   U.S. patent application Ser. No. 11/640,831, entitled “ROLLING        CACHE CONFIGURATION FOR A DATA REPLICATION SYSTEM;”    -   U.S. patent application Ser. No. 11/640,829, entitled “SYSTEMS        AND METHODS FOR MONITORING APPLICATION DATA IN A DATA        REPLICATION SYSTEM;”    -   U.S. patent application Ser. No. 11/640,824, entitled        “DESTINATION SYSTEMS AND METHODS FOR PERFORMING DATA        REPLICATION;” and    -   U.S. patent application Ser. No. 11/640,835, entitled “NETWORK        REDIRECTOR SYSTEMS AND METHODS FOR PERFORMING DATA REPLICATION.”

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present disclosure relates to performing copy and/or data managementoperations in a computer network and, in particular, to systems andmethods for performing data replication in a storage management system.

2. Description of the Related Art

Computers have become an integral part of business operations such thatmany banks, insurance companies, brokerage firms, financial serviceproviders, and a variety of other businesses rely on computer networksto store, manipulate, and display information that is constantly subjectto change. Oftentimes, the success or failure of an importanttransaction may turn on the availability of information that is bothaccurate and current. Accordingly, businesses worldwide recognize thecommercial value of their data and seek reliable, cost-effective ways toprotect the information stored on their computer networks.

Many approaches to protecting data involve creating a copy of the data,such as backing up and/or replicating data on one or more storagedevices. When creating a copy of such data, certain factors aregenerally considered. First, a copy of data should not contain datafiles that are corrupt or terminated improperly. Second, a copy of datashould be current enough to avoid data staleness by avoiding too muchtime between copying such that the copied data is still useful should itbe needed. For certain applications, such as networks that storefinancial transactions, copies a week old may be useless, and much morefrequent copying may be needed.

In an attempt to accommodate such storage requirements, certain systemsthrough all the files in a computer network, or through a selected setof critical files, and check the time information of each file. If datahas been written to the file since the last time the system checked thefile's status, then a copy of the file is sent to a storage system. Oneproblem with such systems is that they typically do not work for datakept in very large files. For example, assuming that a copy could bemade of the very large database, the time needed to make copies of sucha large database may render data shadowing impractical. Making numerouscopies of a large database not only takes a tremendous amount of time,but also requires a tremendous amount of storage space.

Another approach that has been attempted in order to overcome some ofthese limitations is a process whereby a time sequence of data iscaptured and saved. For example, many systems incorporate disk mirroringor duplexing. In disk mirroring or duplexing, changes made to a primarymass storage system are sent to other backup or secondary mass storagesystems. In other words, when a data block is written to the primarymass storage system, the same data block is written to a separatesecondary mass storage system. By copying each write operation to asecond mass storage system, two mass storage systems may be keptsynchronized so that they are virtually identical at approximately thesame time. Because an entire disk volume is being copied, however,mirroring also requires a tremendous amount of storage space andutilizes a large amount of processing resources.

Furthermore, each of the above-described processes for copying orbacking up data can have a significant impact on the source or primarysystem. For example, processing resources of the source system may beexpended in copying data to a destination system rather than being usedto process application requests.

SUMMARY OF THE INVENTION

In view of the foregoing, a need exists for improved systems and methodsfor the copying and/or replication of data in computing systems. Inparticular, a need exists for systems and methods that reduce the impact(e.g., processing load) on a source, or primary, system when performingone or more data management and/or storage operations on data, such as,for example, application-specific data.

In certain embodiments of the invention, systems and methods aredisclosed for performing substantially continuous replication ofapplication-specific data in a computer network environment. In certainembodiments, systems and methods may further perform one or more datamanagement operations on the replicated data in order to avoid burdeningthe source system. For example, one or more storage operations may beperformed on replicated data that represents a recoverable state, or“known good state” of a particular application running on the sourcesystem.

For instance, in certain embodiments, known good replication copies maybe viewed as copies of production volume data. This feature allows amanagement component in the computing system to directly access, copy,restore, backup or otherwise manipulate the replication copies ofproduction data as if the data was the production data of the sourcesystem, thereby improving various system performance characteristicssuch as access time, reducing memory requirements and reducing impact onsource, or client, applications.

In certain embodiments, the replication copies of the production datainclude time information (such as one or more time stamps) thatindicates the client system time when the production data was modifiedand/or when the subject application was in a known good state. Such timestamps are then associated with the replication data and/or copies ofthe replicated data, thereby allowing for the synchronization of timereferences between the production data and copies of the replicateddata.

Accordingly, certain embodiments of the invention may recognize pointswithin application data that represent a known good state of theapplication. This information is then used by the replication system tointelligently replicate sections of application data that represent arecoverable state rather than the rote copying of certain blocks of databased on hardware capacity and/or criteria that are unrelated toapplication recoverability. Thus, in certain embodiments, one benefit ofthe systems and methods provided herein is the ability to replicate dataon an application-specific basis rather than merely copying certainphysical blocks of information based on buffer size, file type orcopying other uncoordinated groups of data.

In certain embodiments, a method is disclosed for performing datamanagement operations in a computer network. The method comprisesmonitoring: operations associated with an application that are operativeto write data to a first storage device; inserting in a log file amarker comprising time information identifying a time of a known goodstate of the application; copying the data to a second storage devicebased at least in part on the operations; generating a snapshot of thecopied data at a time after the data has been copied to the secondstorage device; and associating the snapshot of the copied data with thetime information identifying the time of the known good state of theapplication.

In certain further embodiments, the above-disclosed method additionallycomprises performing a backup or other storage operation on the snapshotand associating the resulting copy of the snapshot with the timeinformation identifying the time of the known good state of theapplication.

In certain embodiments, a system is disclosed for performing datamanagement operations in a computer network environment. The systemcomprises at least one computer application configured to execute on asource computer and a first storage device coupled to the sourcecomputer to receive data write operations from the at least one computerapplication. The system further comprises a second storage device and atleast one module configured to monitor the data write operations and togenerate log entries based on the data write operations, at least one ofthe log entries having a time stamp indicative of a time of a known goodstate of the at least one computer application. The system alsocomprises a replication module coupled to the second storage device,wherein the replication module is configured to: process, based on thelog entries, the data write operations to replicate data to a firstlocation on the second storage device; perform a storage operation onthe replicated data at the first location on the second storage deviceto copy data to a second location different than the first location, andlogically associate the copied data at the second location with the timestamp indicative of the time of the known good state of the at least onecomputer application.

In certain embodiments, a method is disclosed for performing datamanagement operations in a computer network. The method comprisesmonitoring data operations associated with an application, the dataoperations operative to write data to a first storage device andpopulating a first log file with data entries indicative of the dataoperations. The method also comprises: inserting in the first log file amarker indicative of a known good state of the application, the markerincluding a time information identifying a time of the known good stateof the application; replaying to a second storage device the dataoperations based on the data entries to replicate data to a firstlocation on the second storage device; performing a storage operation onthe replicated data to copy the replicated data from the first locationto a second location; and associating the copy of the replicated data atthe second location with the time information identifying the time ofthe known good state of the application.

In certain embodiments, a system is disclosed for copying data in acomputer network. The system comprises: means for monitoring dataoperations generated by a single computer application that are operativeto write data to a first storage device; means for storing data entriesindicative of the data operations; means for inserting in the storingmeans a marker indicative of a known good state of the computerapplication, the marker including a time stamp associated with a sourcesystem time of the known good state of the computer application; meansfor processing, based on the data entries, in a second storage devicethe data operations to replicate data to a first location on the secondstorage device; means for performing a storage operation on thereplicated data in the second storage device to copy the replicated datato a second location; and means for associating the copy of thereplicated data with the time stamp associated with the source systemtime of the known good state of the computer application.

In certain embodiments, a method is disclosed for monitoring a computerapplication to perform data replication in a computer network. Themethod comprises: detecting data operations sent from a computerapplication to a file system on a source computer, the data operationsoperative to write data to a first storage device; populating a firstlog file with data entries indicative of the data operations; detectinga known good state of the computer application; quiescing, at the knowngood state of the computer application, the sending of the dataoperations to the file system; and inserting in the first log file amarker including time information identifying a time of the known goodstate of the computer application.

In certain further embodiments, quiescing of the above-disclosed methodis performed at periodic intervals. For instance, the method may furtherinclude receiving user input for selecting the periodic intervals and/orthe periodic intervals may be selected automatically. For instance, afrequency of the periodic intervals may be based at least in part on: atype of data associated with the computer application, an averagefailure rate of at least one computer in the computer network, a load ofat least one computer in the computer network, an availability of atleast one computer in the computer network, combinations of the same orthe like.

In certain embodiments, a system is disclosed for generating applicationdata for use in a data replication environment. The system comprises amonitor module configured to monitor data write operations of a computerapplication and to generate first log entries based on the data writeoperations, the monitor module being further configured to quiesce orbuffer additional data write operations at a known good state of thecomputer application and to generate a second log entry having a timestamp indicative of a time of the known good state of the computerapplication. The system also comprises at least one log file incommunication with the monitor module and configured to store the firstand second log entries.

In certain embodiments, a system is disclosed for generating applicationdata for use in a data replication environment. The system comprises:means for detecting data operations sent from a computer application toa file system, the data operations operative to modify data stored on afirst storage device; means for storing data entries indicative of thedata operations; means for quiescing, at a known good state of thecomputer application, the sending of additional data operations to thefile system; and means for recording a time stamp identifying a time ofthe known good state of the computer application.

In certain embodiments, a method is disclosed for copying data generatedon a source system in a computer network. The method comprises:processing, with one or more routines, at least one log file having aplurality of log entries indicative of operations generated by acomputer application executing on a source system, the operations beingdirected to data on a source storage device; replaying, with the one ormore routines, the operations on a destination storage device toreplicate application-specific data to the destination storage device;suspending the replaying when the one or more routines encounters aconsistency point marker in the at least one log file, the consistencypoint marker being indicative of a known good state of the computerapplication; and performing a storage operation on the replicated datawhen the replicated data represents the known good state of the computerapplication.

In certain embodiments, a destination system is disclosed for performingdata replication in a computer network. The destination system comprisesat least one replication log file and a replication module. The at leastone replication log file further comprises: (i) a plurality of logentries indicative of data operations generated by a computerapplication for execution on a source storage device and (ii) at leastone consistency point marker indicative of a known good state of thecomputer application. The replication module is configured to replicatedata to a destination storage device and further comprises a replicationagent and at least one process configured to traverse the plurality oflog entries in the at least one replication log file and to copy the logentries to execute the data operations on the destination storagedevice, the at least one thread being further configured to notify thereplication agent when encountering the at least one consistency pointmarker.

In certain embodiments, a system is disclosed for replicating datagenerated on a source device in a computer network. The systemcomprises: means for storing a plurality of log entries indicative ofmodification operations generated by a computer application executing ona source system, the modification operations being directed to data on asource storage device; means for traversing the storing means and forreplaying the modification operations to replicate application-specificdata to a destination storage device; means for suspending the replayingwhen the traversing and replaying means encounters a consistency pointmarker in the means for storing, the consistency point marker beingindicative of a known good state of the computer application; and meansfor performing a storage operation on the replicated data when thereplicated data represents the known good state of the computerapplication.

In certain embodiments, a method is disclosed for handling data to becopied in a computer network. The method comprises: monitoringoperations associated with a single application executing on a sourcesystem; identifying from the operations a plurality of data modificationoperations operative to write data to a first storage device; bufferinga copy of each of the plurality of data modification operations;forwarding the copies of the plurality of data modification operationsto a destination system to be copied to a second storage device withoutfirst writing the copies of the plurality of data modificationoperations to the first storage device; and forwarding the plurality ofdata modification operations to a file system associated with the firststorage device.

In certain embodiments, a system is disclosed for selecting applicationdata to be copied in a computer network. The system comprises a buffer,at least one computer application configured to generate operationsassociated with data on a first storage device, and a filter moduledisposed between the at least one computer application and the firststorage device, the filter module configured to identify from theoperations a plurality of data modification operations. The systemfurther comprises a network redirector component in communication withthe filter module and configured to temporarily store a copy of the datamodification operations in the buffer while allowing the datamodification operations to pass through to the first storage device, thenetwork redirector component being further configured to transmit thecopies of the plurality of data modification operations to a destinationsystem, to be copied to a second storage device, without first writingthe copies of the plurality of data modification operations to the firststorage device.

In certain embodiments, a system is disclosed for acquiring data forreplication on a network storage device. The system comprises means forgenerating operations associated with data on a first storage device andmeans for identifying from the operations a plurality of datamodification operations, the means for identifying being disposedbetween the means for generating and the first storage device. Thesystem further comprises means for temporarily storing a copy of thedata modification operations while allowing the data modificationoperations to pass through to the first storage device, the means forstoring being further configured to transmit the copies of the pluralityof data modification operations to a destination system, to bereplicated to a second storage device, without writing the copies of theplurality of data modification operations to the first storage device.

In certain embodiments, a method is disclosed for performing pathnametranslation in a data replication system. The method comprises:receiving a log entry, to be replicated on a destination system, thatidentifies a data management operation and an inode associated with asource system location corresponding to the data management operation,the inode being one of a plurality of inodes on the source system;accessing a database comprising path data associating each of theplurality of inodes with a short name and a parent inode; constructingfrom the path data an absolute pathname on the destination system thatcorresponds to the inode of the log entry; and forwarding the log entryand the absolute pathname to the destination system.

In certain embodiments, a system is disclosed for performing pathnametranslation during data replication in a computer network. The systemcomprises a database, at least one log entry and a database process. Incertain embodiments, the database comprises path data that associateseach of a plurality of inodes on a source system with a short name and aparent inode. The at least one log entry identifies a data managementoperation and at least one corresponding inode of the plurality ofinodes. The database process is configured to receive the at least onelog entry and to access the database to translate the at least onecorresponding inode to a pathname identifying a location on adestination system corresponding to the data management operation.

In certain embodiments, a system is disclosed for performing translationfrom an inode to a pathname in a data storage network. The systemcomprises: means for storing path data associating each of a pluralityof inodes on a source system with a short name and a parent inode; meansfor identifying data management operations and at least one of theplurality of inodes that corresponds to each data management operation;and means for accessing the storing means to translate the at least oneinode to a pathname identifying a location on a destination systemcorresponding to each data management operation.

In certain embodiments, a method is disclosed for transmitting data froma source computer to a destination computer in a data replicationsystem. The method comprises: monitoring file system requests of anapplication on a source computer; identifying, from the file systemrequests, data management operations directed to data on a first storagedevice; and storing in a buffer a plurality of log entries representingthe data management operations. The method further comprises:sequentially processing each of the plurality of log entries;transmitting each processed log entry to a destination computer;replaying each processed log entry to replicate the data on the firststorage device to a second storage device; and when said sequentiallyprocessing drops below a predetermined rate, disk swapping most recentlyreceived log entries from the buffer to a memory.

In certain embodiments, a system is disclosed for transmitting data tobe replicated in a computer network. The system comprises a queue, abuffer and a monitor module configured to monitor file system requestsgenerated by a computer application, the monitor module being furtherconfigured to populate the queue with log entries indicative of filesystem requests comprising data management operations to be executed ona first storage device. The system further comprises a first threadconfigured to transfer the log entries from the queue to the buffer anda second thread configured to retrieve the log entries from the buffer,process the log entries, and forward the processed log entries to adestination system for replaying the log entries to replicate data on asecond storage device. In certain embodiments, the monitor module isfurther configured to throttle the computer application when a rate atwhich the second thread processes the log entries is below apredetermined rate.

In certain embodiments, a system is disclosed for transmitting data tobe replicated in a network environment. The system comprises: means formonitoring file system requests of an application on a source computerand for identifying, from the file system requests, data managementoperations directed to data on a first storage device; means for storinga plurality of log entries representing the data management operations;means for sequentially processing each of the plurality of log entries;means for transmitting each processed log entry to a destinationcomputer; means for replaying each processed log entry to replicate thedata on the first storage device to a second storage device; and meansfor disk swapping most recently received log entries from the storingmeans to a memory when said sequentially processing falls below apredetermined rate.

In certain embodiments, a system is disclosed for facilitating datasynchronization following a network failure in a data replicationenvironment. The system comprises: a plurality of log entriesrepresenting data management operations generated by an application fordata on a first storage device, each of the plurality of log entriesincluding a unique identifier; a cache memory configured to store aportion of the plurality of log entries such that when a size of theportion of log entries exceeds a storage threshold of the cache memory,the cache memory deletes one or more of the least recently received logentries until the size is less than the storage threshold; at least onedestination computer configured to process the plurality of log entriesto replicate data to a second storage device, the at least onedestination computer being further configured to record the uniqueidentifier of a most recently processed log entry; and at least onereplication routine configured to sequentially retrieve each of theplurality of log entries from the cache memory and to forward each logentry to the at least one destination computer without waiting for anacknowledgement from the at least one destination computer.

In certain embodiments, a method is disclosed for performing datareplication in a computer network. The method comprises: storing logentries in a first-in first-out memory, wherein each of the log entriesincludes a unique identification and represents a data managementoperation generated by an application for data on a first storagedevice; maintaining in the first-in first-out memory a history of thelog entries stored therein such that the storage amount occupied by thehistory of the log entries is less than a storage threshold;transmitting a copy of each log entry from the first-in first-out memoryto at least one destination computer; replaying the copies of the logentries received by the at least one destination computer to replicatethe data management operations on data on a second storage device; andstoring on the at least one destination computer the uniqueidentification of the most recently replayed log entry.

In certain embodiments, a system is disclosed for facilitating datasynchronization following a network failure in a data replicationenvironment. The system comprises: means for representing datamanagement operations generated by an application for data on a firststorage device, each of the means for representing including a uniqueidentifier; means for storing a portion of the means for representingsuch that when a size of the portion of the means for representingreaches a storage threshold of the means for storing, the means forstoring deletes one or more of the oldest means for representing untilthe size is less than the storage threshold; means for processing themeans for representing to replicate data to a second storage device, themeans for processing being further configured to record the uniqueidentifier of a most recently processed means for representing; andmeans for sequentially retrieving log entries from the means forstoring, the means for retrieving being further configured to forwardeach means for representing to the means for processing without waitingfor an acknowledgement from the means for processing.

In certain embodiments, a system is disclosed for identifying andreplicating software application data representative of a known goodstate. The system comprises a data agent associated with a softwareapplication that identifies data to be replicated indicative of theknown good state, wherein the data representative of the known goodstate is indicated by a consistency point, or marker, in the data. Thesystem further includes a replication module that copies the datarepresentative of the known good state to a storage device based on theconsistency point.

According to certain embodiments, a system and method are provided forcreating consistency points in a replication volume, whereby thereplication volume may be created by application-specific replicationprocesses. In other words, the user may establish a storage policy thatidentifies storage operations that may be performed using differenttypes of replication copies. In some instances, it may not beadvantageous to replicate an entire volume. For example, viewable datamay be the only data that is replicated (e.g., SQL data). Therefore, anapplication-specific storage policy associated with replication may becreated in combination with journal (e.g., all file system logs) loggingoperations for the purposes of generating application-specificreplication.

For purposes of summarizing the disclosure, certain aspects, advantagesand novel features of the inventions have been described herein. It isto be understood that not necessarily all such advantages may beachieved in accordance with any particular embodiment of the invention.Thus, the invention may be embodied or carried out in a manner thatachieves or optimizes one advantage or group of advantages as taughtherein without necessarily achieving other advantages as may be taughtor suggested herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a replication system according tocertain embodiments of the invention.

FIG. 2A illustrates a block diagram of an exemplary embodiment of asource system of the replication system of FIG. 1.

FIG. 2B illustrates a block diagram of another embodiment of a sourcesystem usable with the replication system of FIG. 1 and having a networkredirector module.

FIG. 3 illustrates a block diagram of an exemplary embodiment of adestination system of the replication system of FIG. 1.

FIG. 4 illustrates a block diagram of further details of an exemplaryembodiment of the replication system of FIG. 1.

FIG. 5 illustrates various fields of an exemplary embodiment of a logentry usable by the replication systems of FIGS. 1 and 4.

FIG. 6 illustrates a block diagram of an exemplary embodiment of thereplication system of FIG. 1 having a storage manager module.

FIG. 7 illustrates a flow chart of an exemplary embodiment of aninstallation process usable by the replication system of FIG. 4.

FIG. 8 illustrates a flow chart of an exemplary embodiment of areplication process usable by the replication system of FIG. 4.

FIG. 9 illustrates a block diagram of another embodiment of areplication system having multiple replication modules and destinationstorage devices.

FIG. 10 illustrates a block diagram of an exemplary embodiment of arolling cache configuration usable with embodiments of a replicationsystem.

FIG. 11 illustrates a block diagram of an exemplary embodiment of apathname translation system usable with embodiments of a replicationsystem.

FIG. 12 illustrates an exemplary embodiment of a pathname translationdatabase usable with embodiments of the pathname translation system ofFIG. 11.

FIG. 13 illustrates a flowchart of an exemplary embodiment of a pathnametranslation process executable by the pathname translation system ofFIG. 12.

FIGS. 14A and 14B illustrate block diagrams of exemplary embodiments ofuser interface portions usable with embodiments of a replication system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

As will be seen from the disclosure herein, certain embodiments ofsystems and methods are provided for intelligent data replication. Inparticular, embodiments of the invention include the replication ofapplication-specific data from a source system to a destination system.For example, one or more modules executing on the source system maymonitor data management operations (such as data modificationoperations) of a specific application and generate log entriesindicative of the operations. The log entries may then be copied to andprocessed, or replayed, by a destination system to replicate data in oneor more replication volumes, the replicated data being a copy of theapplication data stored on a source storage device.

In certain embodiments, the replication systems and methods disclosedherein are further configured to identify a known good, or recoverable,state of the application. For instance, the replication systems andmethods may determine appropriate points within certain application dataat which certain information is collected to reliably restore theapplication as of a particular point in time. At certain known goodstates, the replication system quiesces the application and/or generatesa consistency point marker having time information, such as, forexample, a time stamp, indicating the source system time of theapplication's known good state.

When replicating the data, the destination system utilizes theconsistency point markers to identify when the replicated datarepresents the known good state of the application. The destinationsystem is further capable of performing one or more data managementoperations, such as, for example, storage operations (e.g., snapshot,backup), search operations, data classification, combinations of thesame or the like on the replicated data at certain consistency points.Performing data management operations on the replicated data allows forthe processing of copies of application data without significantlyimpacting the resources of the source system. Furthermore, when copyingthe replicated data at consistency points, the copied data presumablyrepresents a known good state of the application.

Using the time information of one or more consistency point entries, thedestination system is further capable of logically associating thecopied data (e.g., a snapshot of the replicated data) with the sourcesystem time of the known good state of the application. That is, eventhough the copy of the replicated data occurs at some point after thereplication of the production (source system) data, the copied data isassociated with the (earlier) source system time of the consistencypoint entry (e.g., the source system time of the known good state of theapplication. As a result, the copy of the replicated data, which islogically associated with the source system time of the known good stateof the application, appears as if the copy was directly performed on theproduction (source system) data. In certain embodiments, this methodadvantageously allows for further processing of the copied data (e.g.,processing of snapshots of the replicated data) without touching thedata of the source system.

The features of the systems and methods will now be described withreference to the drawings summarized above. Throughout the drawings,reference numbers are re-used to indicate correspondence betweenreferenced elements. The drawings, associated descriptions, and specificimplementation are provided to illustrate embodiments of the inventionand not to limit the scope of the disclosure.

FIG. 1 illustrates a block diagram of a replication system 100 accordingto certain embodiments of the invention. As shown, the replicationsystem 100 comprises a source system 102 capable of communicating with adestination system 104 by sending and/or receiving data over a network106. For instance, in certain embodiments, the destination system 104receives and/or stores a replicated copy of at least a portion of data,such as application-specific data, associated with the source system102.

The illustrated network 106 advantageously comprises any means forcommunicating data between two or more systems or components. It certainembodiments, the network 106 comprises a computer network. For example,the network 106 may comprise a public network such as the Internet,virtual private network (VPN), token ring or TCP/IP based network, widearea network (WAN), local area network (LAN), an intranet network,point-to-point link, a wireless network, cellular network, wireless datatransmission system, two-way cable system, interactive kiosk network,satellite network, broadband network, baseband network, combinations ofthe same or the like. In embodiments wherein the source system 102 anddestination system 104 are part of the same computing device, thenetwork 106 may represent a communications socket or other suitableinternal data transfer path or mechanism.

As shown, the source system 102 comprises one or more applications 108residing on and/or being executed by a computing device. For instance,the applications 108 may comprise software applications that interactwith a user to process data and may include, for example, databaseapplications (e.g., SQL applications), word processors, spreadsheets,financial applications, management applications, e-commerceapplications, browsers, combinations of the same or the like. Forexample, in certain embodiments, the applications 108 may comprise oneor more of the following: MICROSOFT EXCHANGE, MICROSOFT SHAREPOINT,MICROSOFT SQL SERVER, ORACLE, MICROSOFT WORD and LOTUS NOTES.

The source system 102 further comprises one or more processes, such asfilter drivers 110, that interact with data (e.g., production data)associated with the applications 108. For instance, the filter driver110 may comprise a file system filter driver, an operating systemdriver, a filtering program, a data trapping program, an application, amodule of the application 108, an application programming interface(“API”), or other like software module or process that, among otherthings, monitors and/or intercepts particular application requeststargeted at a file system, another file system filter driver, a networkattached storage (“NAS”), a storage area network (“SAN”), mass storageand/or other memory or raw data. In same embodiments, the filter driver110 may reside in the I/O stack of the application 108 and mayintercept, analyze and/or copy certain data traveling from theapplication 108 to a file system.

In certain embodiments, the filter driver 110 may intercept datamodification operations that include changes, updates and newinformation (e.g., data writes) with respect to the application(s) 108of interest. For example, the filter driver 110 may locate, monitorand/or process one or more of the following with respect to a particularapplication 108, application type or group of applications: datamanagement operations (e.g., data write operations, file attributemodifications), logs or journals (e.g., NTFS change journal),configuration files, file settings, control files, other files used bythe application 108, combinations of the same or the like. In certainembodiments, such data may also be gathered from files across multiplestorage systems within the source system 102. Furthermore, the filterdriver 110 may be configured to monitor changes to particular files,such as files identified as being associated with data of theapplications 108.

In certain embodiments, multiple filter drivers 110 may be deployed on acomputing system, each filter driver being dedicated to data of aparticular application 108. In such embodiments, not all informationassociated with the client system 102 may be captured by the filterdrivers 110 and, thus, the impact on system performance may be reduced.In other embodiments, the filter driver 110 may be suitable for use withmultiple application types and/or may be adaptable or configurable foruse with multiple applications 108. For example, one or more instancesof customized or particularizing filtering programs may be instantiatedbased on application specifics or other needs or preferences.

The illustrated source system 102 further comprises a source storagedevice 112. The source storage device 112 may include any type of mediacapable of storing data. For example, the source storage device 112 maycomprise magnetic storage (such as a disk or a tape drive) or other typeof mass storage. In certain embodiments, the source storage device 112may be internal and/or external to (e.g., remote to) the computingdevice(s) having the applications 108 and the filter drivers 110.

As further illustrated in FIG. 1, the destination system 104 comprises areplication module 114 and a destination storage device 116. In certainembodiments, the replication module 114 is configured to monitor and/ormanage the copying of data from the source system 102 to the destinationsystem 104, such as data retrieved by the filter drivers 110. In yetother embodiments, the replication module 114 is a “dumb” server orterminal that receives and executes instructions from the source system102.

The destination storage device 116 may include any type of media capableof storing data, such as replication data sent from the source system102. For example, the destination storage 116 device may comprisemagnetic storage (such as a disk or a tape drive) or other type of massstorage. In certain embodiments, the destination storage device 116 maybe internal and/or external to the computing device(s) having thereplication module 114.

In certain embodiments, the source storage device 112 and/or thedestination storage device 116 may be implemented as one or more storage“volumes” that include physical storage disks defining an overalllogical arrangement of storage space. For instance, disks within aparticular volume may be organized as one or more groups of redundantarray of independent (or inexpensive) disks (RAID). In certainembodiments, either or both of the storage devices 112, 116 may includemultiple storage devices of the same or different media.

FIG. 2A illustrates a block diagram of an exemplary embodiment of thesource system 102 of FIG. 1. In particular, the source system 102comprises a client computer 230 on which the application(s) 108 and thefilter driver(s) 110 reside and/or are executed. In certain embodiments,the client computer 230 comprises any computing device capable ofprocessing data and includes, for example, a server computer, aworkstation, a personal computer, a cell phone, a portable computingdevice, a handheld computing device, a personal digital assistant (PDA)or the like.

The illustrated client computer 230 further comprises a file system 234for organizing files and directories accessible by the client computer230. In certain embodiments, the file system 234 comprises a datastructure usable to keep track of a collection of files and/ordirectories stored on the source storage device 112. The file system 234may include, for example, a local file system, a network file system, afile server, a management program or the like, or may include multiplefile systems accessible by an operating system. For instance, inembodiments wherein the storage device 112 is associated with multiplevolumes, each volume may be associated with its own file system 234, ora single file system 234 may span across the multiple volumes.

In certain embodiments, the file system 234 comprises a “write in-place”file system, an example of which is the “fast file” system. In a writein-place file system, the locations of the data structures, such as datablocks, and other descriptive information on disk are typically fixed.Certain data structures are used to store information (e.g., metadata)about a file, whereas the data blocks are structures used to store theactual data for the file. The information contained in certain datastructures may include, for example, ownership of the file, accesspermission for the file, size of the file, file type, references tolocations on disk of the data blocks for the file, combinations of thesame or the like. The references to the locations of the file data maybe provided by pointers, which may further reference indirect blocksthat, in turn, reference the data blocks, depending upon the quantity ofdata in the file. In certain embodiments, changes are made “in-place” inaccordance with the write in-place file system. If an update to a fileextends the quantity of data for the file, an additional data block isallocated and the appropriate management files are updated to referencethat data block.

The illustrated client computer 230 also comprises one or more dataagents 236. In certain embodiments, the data agent 236 comprises amodule responsible for performing data and/or storage tasks related tothe client computer 230. For example, the data agent 236 may manageand/or coordinate the compilation of and/or transferring of replicationdata from the source system 102. In other embodiments, the data agent236 may provide archiving, migrating, and/or recovery of client computerdata.

In certain embodiments, the client computer 230 comprises a plurality ofdata agents 236, each of which performs data management operationsrelated to data associated with each application 108. In suchembodiments, the data agent 236 may be aware of the various files,folders, registry files and/or system resources that are impacted by aparticular application 108. For instance, the data agent 236 may beprogrammed to detect data management requests by a particularapplication 108 and determine which files, folders and/or systemresources are associated with the data management requests.

In certain embodiments, different individual data agents 236 may bedesigned to handle MICROSOFT EXCHANGE data, LOTUS NOTES data, MICROSOFTWINDOWS 2000 file system data, MICROSOFT ACTIVE DIRECTORY OBJECTS data,and other types of data. In certain further embodiments, one or moredata agents 236 may be configured to backup, migrate, and/or recoverapplication-specific data.

For example, in embodiments wherein the source system 102 comprises aMICROSOFT EXCHANGE 2000 server, the source system 102 may use multipledata agents 236 to perform storage operations (e.g., backup, migrate,replication or restore operations). For instance, a MICROSOFT EXCHANGE2000 mailbox data agent may be used to replicate EXCHANGE 2000 mailboxdata; a MICROSOFT EXCHANGE 2000 database data agent may be used toreplicate EXCHANGE 2000 database data; a MICROSOFT EXCHANGE 2000 publicfolder data agent may be used to replicate EXCHANGE 2000 public folderdata; and a MICROSOFT WINDOWS 2000 file system data agent may be used toreplicate file system data.

In certain embodiments, multiple data agents 236 may be treated asseparate data agents even though they reside on the same client computer230. In other embodiments, the separate data agents may be combined toform a virtual data agent for performing storage operations related to aspecific application. Thus, the four separate data agents of the aboveexample may be combined as a virtual data agent suitable for performingstorage operations related to MICROSOFT EXCHANGE 2000 data.

In certain embodiments, the data agent 236 is configured to perform datamanagement operations in accordance with one or more “storage policies”or other preferences. A storage policy may include a data structure orother information having a set of preferences and other storage criteriafor performing a storage operation. The preferences and storage criteriamay include, but are not limited to, information regarding storagelocations, relationships between system components, network pathways,retention policies, data characteristics, compression or encryptionrequirements, preferred system components, combinations of the same orthe like.

In certain embodiments, one or more data agents 236 are configured toperform an initial “seeding” process of a replication process. Forexample, prior to (or concurrently with) data replication using one ormore filter drivers 110, the data agent 236 may perform a scan of thesource system 102 (e.g., the source storage device 112). For instance,the data agent 236 may evaluate the folders and/or directory structureof the source system 102 to determine which folders are used by aparticular application 108. In certain embodiments, the data agent 236may also identify, arrange, and queue necessary data of the application108 to provide a proper platform for replication. For example, the dataagent 236 may populate source log(s) 244 with application data that hasalready been written to the source storage database 112.

In certain embodiments, when the data agent 236 is initially installedor enabled on the client computer 230, the data agent 236 may evaluatethe application 108. For instance, the data agent 108 may determine theapplication's organizational structure, which may include, for example,folder, directory and file information. The information gathered by thedata agent 236 may be sufficient to define a complete “set” ofinformation to be replicated such that suitable baseline datarepresenting the current operational state of the application 108 isidentified. In some instances, this initial process may require theexamination and identification of data related to application operationsoccurring prior to the installation of data agent 236. The data agent236 may also be configured to identify general configuration andoperational information regarding the application 108. In certainembodiments, the data agent 236 may be configured to access and/ormonitor particular files, folders, directories, registries, preferencesand/or other like data structures for information to be replicated. Allor a portion of the information gathered by the data agent 236 may becopied over to the destination system 104 as part of the initial seedingor initialization process. After the seeding process is complete, datareplication may occur on a substantially continuous basis based on datatransfers occurring between application(s) 108 and source storage device112. In certain embodiments, the seeding process may occur substantiallyconcurrently with execution of the application(s) 108. For instance,data operations from the application(s) 108 may be temporarily stored ina queue or buffer until the seeding process, or a portion thereof, iscomplete.

In certain embodiments, the data agent 236 may be configured to evaluatean application 108, and based on certain system management parametersand/or other considerations associated with the application 108 (e.g.,data size, frequency of replication, system or user preferences, etc.),the data agent 236 may “map” or correlate the application data to one ormore locations on the destination storage device 116. In otherembodiments, the data agent 236 may communicate with other systemcomponents when making correlation decisions. For example, the dataagent 236 may communicate with the replication agent 356 and/or anoptional storage manager component when deciding how to map particularapplication data.

For example, the data agent 236 may map a certain application 108 to onelocation on the destination storage device 116, or may parse or separateapplication data for storage across multiple volumes of the destinationstorage device 116 depending on preferences or system managementobjectives. Parsing of data across multiple volumes may occur based onapplication type (certain applications spread across multiple volumes),data type, temporal considerations (e.g., data for a specified timeperiod is stored on a particular volume), size considerations (e.g.,data up to a certain size is stored on one volume), relative importanceof data, conservation of memory space, combinations of the same or thelike. Any suitable parsing criteria may be used if desired to facilitaterecall, storage or management of application data.

As shown in FIG. 2A, the client computer 230 communicates through thefile system 234 with the source storage device 112, which furtherincludes a database 240 and database logs 242. In yet other embodiments,the client computer may communicate with NAS or the like. In certainembodiments, data referred to the source storage device 112 may be firstwritten to a file in the database logs 242 and subsequently committed tothe database 240 in accordance with data management techniques forenhancing storage operation performance. Moreover, although only onedatabase 240 and one database log 242 are depicted in FIG. 2A, it willbe understood that the source storage device 112 may comprise additionaldatabases 240, database logs 242 and/or other directory and file storagestructures to meet the storage needs of the client computer 230.

As illustrated in FIG. 2A, the filter driver 110 is advantageouslylocated between the application 108 and the file system 234. Forinstance, the filter driver 110 may be deployed in the stack as an I/Obuffer and/or process in the data path between the application 108 andthe file system 234. In such embodiments, the filter driver 110 mayintercept, snoop, supervise, trap, process or otherwise be cognizant ofsome or all operations (e.g., data modification operations, filemodification operations, read operations and the like) from theapplication 108 to its associated location(s) on the source storagedevice 112.

For example, in certain embodiments, the filter driver 110 maycommunicate with an associated data agent 236 to determine where datafor a particular application 108 will be stored (e.g., particularfolders on the file system 234). In certain embodiments, the filterdriver 110 and/or the data agent 236 may also monitor and/or parse datamanagement operations to determine if new or additional folders areaffected by the production volume data of the particular application108. In certain embodiments, the data agent 236 may monitor datamanagement operations and/or other data for other purposes, such as, forexample, for satisfying a query or command by a storage managercomponent or the like.

As further depicted in FIG. 2A, one or more of the filter drivers 110and associated data agent(s) 236 may be grouped together as a singlemodule, such as driver module 237. In yet other embodiments, the dataagent(s) 236 may be separate from the driver module 237.

As discussed above, in certain embodiments, the filter driver 110 ispreferably configured to monitor and/or filter data managementoperations associated with a particular application 108. The filterdriver 110 may be further configured, according to predefined criteria,to cause particular data to be written to one or more source logs 244for subsequent replication. For instance, the filter driver 110 may beconfigured to intercept, scrub, parse and/or trap data managementoperations and to populate the source logs 244 with changes associatedtherewith.

In certain embodiments, the filter driver 110 may examine the datamanagement operation in progress, determine whether the type ofoperation is one of interest for replication purposes, and/or copyselect or all data to source log 244. For instance, as discussed above,the filter driver 110 may determine if the data management operationconcerns data in one or more files determined as relevant to replication(e.g., files that may store data for a particular application). In otherembodiments, the filter driver 110 may generate log entries for all datamanagement operations.

The filter driver 110 may further process and/or traverse the data andcopy, generate or examine other relevant information, such as a logentry number, time information (e.g., time stamp), application type,data size and start field, combinations of the same or the like, thatmay be useful in the replication process. In other embodiments, thefilter driver 110 may monitor files on the source storage device 112 formodifications of data relating to the subject application 108. Forinstance, as disclosed above, the filter driver 110 may monitor a selectgroup of files, which have been associated with the application 108, orfolders to detect changes to data stored therein. In certainembodiments, the filter driver 110 or other system component may detectwhen a data write operation of the application is made to a file orfolder not in the select group. The filter driver 110 or other systemcomponent may then determine from the properties of the data writemodification if the subject folder or file should be added to the selectgroup (for subsequent monitoring).

In certain embodiments, the filter driver 110 is deployed (e.g., by dataagent 236) on the client computer 230 prior to the beginning of thereplication process. In embodiments wherein the filter driver 110 isdeployed after replication begins, pertinent application data alreadystored on the source storage device 112 may be copied to the source logs244 prior to the replication process (e.g., during the initial “seeding”process described above).

In certain embodiments, the filter driver 110 may be enabled and/ordisabled by the data agent 236. For instance, enabling the filter driver110 may allows it to populate an associated source log 244 with logentries from application data passed from the application 108 to thesource storage device 112. When the filter driver 110 is disabled, datamay pass directly through to the source storage device 112 without beingcopied to the source logs 244.

The source log 244 comprises any type of memory capable of storing oneor more journal or log entries. In certain embodiments, the source log244 comprises a cache memory. The source log 244 may reside on thesource system 102, such as, for example, on the source storage device112, or at least a portion of the source log 244 may be external to thesource system 102.

In certain embodiments, the data agent 236 monitors the storage capacityof the source logs 244. For instance, when one or more of the sourcelogs 244 reach a particular memory threshold, the data agent 236 mayopen a socket and communicate to the destination system 104 that a copyof the source log 244 is ready to be transmitted. In other embodiments,the data agent 236 may be configured to copy the source log 244 to thedestination system 104 at periodic intervals or in accordance with otherpredefined criteria.

Although the source system 102 has been described above with referenceto particular embodiments, other embodiments of the invention mayinclude more or fewer components than those depicted in FIG. 2A. Forexample, FIG. 2B illustrates a block diagram of a source system 102′according to certain embodiments of the invention. As shown, the sourcesystem 102′ comprises the client computer 230, application 108, filterdriver 110, file system 234, source log 244, and source storage device112 (which includes database 240 and database log 242).

The illustrated source system 102′ further comprises a networkredirector module 246 located between the filter driver 110 and the filesystem 234. In certain embodiments, the network redirector module 246 isadvantageously capable of improving performance of the data replicationsystem. In particular, the network redirector module 246 allows for adata management operation to be sent to a destination system (e.g.,destination system 104 of FIG. 1) prior to and/or substantiallyconcurrently with the data management operation being forwarded to thesource storage device 112.

For example, with reference to FIG. 2B, when the application 108 issuesa data write operation, the filter driver 110 identifies whether or notthe data write operation is one of interest, such as according toparticular criteria established by the data agent 236 and/or a storagemanager module. If so, the filter driver 110 notifies the networkredirector module 246.

The illustrated network redirector module 246 is configured to buffer acopy of the data write operation to a memory 248. For instance, incertain embodiments, the memory 248 comprises a queue, such as afirst-in first-out queue, for receiving copies of the data writeoperations. In certain preferred embodiments, the memory 248advantageously comprises a cache memory, such as a random access memory(RAM). In certain further embodiments, the memory 248 comprises amemory-mapped file that is shared with the filter driver 110 such thatthe filter driver 110 provides a copy of the data write operationdirectly to the memory 248.

In certain embodiments, the network redirector module 246 comprises adriver, such as a file system driver or an operating system driver, thatsends data to a particular destination, such as a remote device (e.g., adestination system, a log, or the like). For instance, the networkredirector module 246 may operate entirely in a kernel mode and may sitabove the transport layer of a TCP/IP protocol. Such a configurationallows the network redirector module 246 to interact directly with thefilter driver 110 without the need for user-to-kernel mode andkernel-to-user mode context switches.

In certain embodiments, the network redirector module 246 is furtherconfigured to access the memory 248 and transmit the copy of the datawrite operation to one or more destination systems through a network(e.g., network mirroring). For instance, the particular destinationsystem may include a counterpart network component that receives thedata write operation from the network redirector module 246 and forwardsthe replicated data write operation to a second storage device (e.g.,destination storage device 116 of FIG. 1). During this time, the othercopy of the data write operation from the application 108 is forwardedto the source storage device 112 for execution.

In certain embodiments, the network redirector module 246 improvesperformance of the source system 102 because data management operationsare able to quickly pass through to the file system 234. That is, thenetwork redirector module 246 may quickly cache a copy of the datamanagement operation rather than writing a copy of the operation to disk(e.g., in the source log 244) prior to forwarding the operation to thesource storage device 112.

In certain embodiments, the source system 102 communicates with theassociated destination system to verify that the two systems aresynchronized. For instance, the source system 102 may receive from thedestination system an identification (e.g., unique serial number) of thedata write operation currently being replicated by the destinationsystem. The source system 102 may then compare the receivedidentification with data write operation being forwarded to the sourcestorage device 112.

FIG. 3 illustrates a block diagram of an exemplary embodiment of thedestination system 104 of FIG. 1. In particular, the destination system104 comprises the replication module 114, which communicates with one ormore replication logs 352 and the destination storage device 116. Incertain embodiments, the replication module 114 comprises any computingdevice capable of processing data and includes, for example, a servercomputer, a workstation, a personal computer or the like.

In certain embodiments, the replication logs 352 contain a copy of thedata stored on the source logs of a client system, such as the sourcelogs 244 of FIG. 2A. The replication logs 352 comprise any type ofmemory capable of storing data including, for example, cache memory. Incertain embodiments, the replication logs 352 may reside on thedestination system 104, such as, for example, on the destination storagedevice 116, or at least a portion of the replication logs 352 may beexternal to the destination system 104. In certain embodiments, once thereplication logs 352 have been populated with the data from the sourcelogs 244, the data on the source logs 244 is available to be erasedand/or overwritten to conserve memory space.

The replication module 114 of the destination system 104 furthercomprises a replication agent 356 and one or more processes, such asthreads 358. In certain embodiments, the replication agent 356 comprisesone or more software modules that coordinate the transfer of data fromthe replication logs 352 to the destination storage device 116.

For example, in certain embodiments, the replication agent 356instantiates an appropriate number of threads, processes, or routines,358 for copying data from the replication logs 352 to the destinationstorage device 116. In certain embodiments, the number of threads 358 isbased on one or more of the following factors: the number of logs filessent from the source logs 244 to the replication logs 352, informationreceived from the data agent(s) 236, information generated by the filterdriver(s) 110, and the type(s) of application data being tracked.

In certain embodiments, the replication agent 356 further includesmapping or correlation information that determines when and to where thedata from the replication logs 352 is copied by the threads 358. Incertain embodiments, such mapping information may be based on system- oruser-defined parameters and/or may be automatically generated, such asbased on the status of the destination storage device 116.

The one or more threads 358 (or processes) direct movement of data fromreplication logs 352 to the appropriate location on the destinationstorage device 116. In operation, in certain embodiments, the threads358 advantageously process (or traverse) replication logs 352 forparticular types of data and then copy that data to certain locations onone or more replication volumes based on data paths identified by thereplication agent 356 and/or associated with each thread 358. Forexample, the thread(s) 358 may sequentially process each entry in thereplication log 352 and write the associated data to the destinationstorage device 116.

In certain embodiments, one thread 358 may write to one or more volumesof the destination storage device 116 and/or multiple threads 358 maywrite to a single volume in parallel. Furthermore, one thread 358 mayaccess one or more replication logs 352, and/or multiple threads 358 mayaccess the same replication log 352.

In certain embodiments, each thread 358 is assigned to a hard-coded pathpair, which includes (i) a source path identifying the location on thesource storage device 112 associated with a data management operation(e.g., c:\Folder\) and (ii) a destination path identifying the locationon the destination storage device 116 to receive the replicated data(e.g., D:\folder\) from the thread 358.

FIG. 4 illustrates further details of a replication system 400 inaccordance with certain embodiments of the invention. As illustrated,the replication system 400 comprises the source system 102 incommunication with the destination system 104, portions of which aredescribed in more detail with respect to FIGS. 1-3.

As detailed above, in certain embodiments, the filter driver 110preferably substantially continuously populates data relating to one ormore of the applications 108 to the source logs 244. As shown in FIG. 4,the source logs 244 further comprise a first log file 460 and a secondlog file 462. In certain embodiments, the filter driver 110 sequentiallywrites log entries to the source logs 244, and when a certain capacityof the first log file 460 is reached, the filter driver 110 beginspopulating the second log file 462 with log entries.

In yet other embodiments, data relating to each application 108 ofinterest may be written to a particular log file established for thatapplication. For example, with reference to FIG. 4, the first log file460 may relate to a first application of interest, whereas the secondlog file 462 may relate to a second application of interest.

In certain embodiments, each of the log files of the source logs 244 maybe established by the data agent(s) 236 and/or the filter driver(s) 110as part of an initial deployment or initialization process. Moreover,data may be written to the source logs 244 as determined by preferencesstored on or accessed by the client computer 230 in a preferencedatabase 465.

For example, as further shown in FIG. 4, the first and second log files460, 462 may comprise a series of entries, each having an identifierthat indicates the sequence order and/or type of entry being made. Forinstance, the illustrated entry identifier (“L1”) may indicate that theparticular entry represents a first database entry in a particular orderof operation. The illustrated entry identifier (“L2”) may indicate asecond database entry in a particular order of operation, and so forth.The illustrated entry identifier (“D1”) may indicate that the particularentry represents a first database commit entry in a particular order ofoperation. Thus, in the example described above, the log entriesidentified by L1 and L2 may correspond to modifications associated witha particular database transaction, and the log entry identified by D1may correspond to a commit command for the particular transaction.

In certain embodiments, the log entries are populated into the log files460, 462, sequentially such that the relative order of the log entriescorresponds generally to the order in which the associated datamanagement operations were performed by the application 108. In otherembodiments, the log entries may be populated into the log files 460,462, in a non-sequential manner.

In certain embodiments, filter driver log entries may be copies orportions of some or all of the data operations outputted fromapplication 108 to the source storage device 112. In yet otherembodiments, the filter driver 110 and/or data agent 236 may constructlog entries based solely on information available in the data streambetween application 108 and the source storage device 112. In yet otherembodiments, the log entries may include other system information, suchas time stamp information. In still other embodiments, the log entriesmay be enriched with other useful system or application informationavailable on the client computer 230 or within the client system 102.For example, the log entries may include metadata and/or dataclassification information that has been obtained from application data.

It will be understood that, although only two log files are shown inFIG. 4, more or fewer log files may be used with embodiments of theinvention. For instance, multiple applications 108 may be monitored bythe filter drivers 110 and, thus, additional log files may be added asnecessary or desired. Moreover, although in some embodiments, eachapplication 108 and each log file in the source logs 244 may have itsown associated filter driver 110, in other embodiments, a single filterdriver 110 may be deployed and configured for use with multipleapplications 108 such that there are separate log files for eachmonitored application 108.

Additionally, it will be understood that it is not necessary fordifferent log files to be established for each application 108. Rather,in certain embodiments, one or more “global” log files may be used, withdata for each application 108 being placed in the global log file andmarked as belonging to a particular application 108. With thisarrangement, the replication system 400 may differentiate among dataassociated with different applications 108 based on data in one or morefields of the log entries, as is discussed in more detail below withrespect to FIG. 5.

With continued reference to FIG. 4, in certain embodiments of theinvention, the data agent 236 and/or filter driver 110 may beadvantageously configured to pause, or quiesce, the application 108during data replication. For instance, the data agent 236 may cause theapplication 108 to temporarily suspend data management operations to thesource storage device 112 once the application 108 reaches a known“good,” “stable” or “recoverable” state. In certain embodiments, such astate may be defined as when particular computing operations of theapplication 108 are complete to a point such that further operation,recovery and/or rolling back of the application 108 may occur, based onthe recorded data, without the loss of critical information or computingoperations needed for operation of the application 108. This point ofreferential integrity is generally referred to herein as a known goodstate of the application 108.

In certain embodiments, the data agent 236 instructs the quiescing ofthe application 108 through an application programming interface (API).For instance, the data agent 236 may send a command (e.g., FLRSNAP.FOO)to the application 108 that causes the application 108 to quiesce. Whenthe application 108 has placed itself in a known good state, theapplication 108 may send an acknowledgment to the data agent 236.

In certain embodiments, once the data management operations aresuspended, the I/O buffers in the data path of the application areflushed (and/or the writes in the queues are flushed), and the sourcelogs 244 are populated. For example, some or all of the pending datamanagement operations (e.g., as of the time of the suspension of theapplication) may be allowed to complete and/or percolate through thedata path. The filter driver 110 and/or data agent 236 then inserts alogical marker or tag in the source log file denoting that a“consistency point” or “consistency recovery point” has been reached. Insome embodiments, the consistency point indicates the time at which theapplication 108 is at a known good state. For instance, in certainembodiments, the data agent 236 instructs the filter driver 110 toinsert a consistency point entry into the source logs 244.

FIG. 4 illustrates consistency point entries as log entries 463 and 464in, respectively, the first and second log files 460, 462. As shown, theconsistency point entries are represented by “CRP” in the source logs244. In certain embodiments, once the consistency point is identifiedand inserted into the source log 244, the data agent 236 may instructthe application 108 to “restart” so as to resume normal data managementoperations from the application 108 to the source storage device 112. Incertain embodiments, the log entries are sequentially written to thesource logs 244 until each consistency point.

Notwithstanding the foregoing, it will be understood that, in certainembodiments, although application 108 is quiesced, it need not actuallypause or suspend operation during the quiescent period. Rather, theapplication 108 may continue to operate substantially normally but mayqueue internally, or otherwise buffer, data management operationsintended for the source storage device 112. After the quiescent period,the buffered modification operations may be allowed to complete (i.e.,be sent to the source storage device 112).

In certain embodiments of the invention, the application 108 isperiodically quiesced (and a corresponding consistency point placed inthe source log 244) based on particular criteria. For instance, thequiescing of the application 108 may be based on one or more system- oruser-defined preferences (e.g., every five minutes). The periodicquiescing of the application 108 may be based on the desired frequencyof performing replication, backup or other data modification operationson the subject data. For instance, applications 108 dealing withdata-sensitive information may necessitate more frequent quiescing (andcreation of consistency points) than other types of applications.

In yet other embodiments, policies for the frequency of consistencypoint entries may be automatically generated. For instance, the dataagent 236 may be configured to quiesce the application 108 based on thestatus (e.g., capacity) of the source logs 244, the replication logs 352and/or the destination storage device 116. In yet other embodiments,quiescing of the application 108 may be performed based on an automaticreporting procedure. For instance, a module of the replication system400 may be configured to gather, receive and/or analyze informationassociated with a failure rate and/or health of applicable servers.Additional details of such status monitoring are provided in U.S. patentapplication Ser. No. 11/120,619, filed May 2, 2005, now published as US2006-0053261 A1, which is hereby incorporated herein by reference in itsentirety. For example, the frequency of consistency points may beselected or adjusted to mitigate risks detected in a storage network. Incertain embodiments, one or more management components of thereplication system 400 may identify a storage-related characteristic andmay modify the frequency of consistency points if the storage-relatedcharacteristic satisfies one or more particular risk criteria. Forinstance, risk criteria may be indicated by storage policies, storagedefinitions, a service level agreement (“SLA”) or other informationlocated in a management component database or other component of thereplication system 400. In certain further embodiments, the frequency ofconsistency points may be based at least in part on a level of dataprotection or data availability in the replication system 400.

In certain embodiments, one or more log entries in the source logs 244are preferably associated with time information, such as, for example,assigned a time stamp indicative of the client system time with whichthe particular log entries are associated. For instance, the timeinformation may indicate the time at which: the log entry is written tothe source log 244, the data management operation is generated by theapplication 108, the data modification operation is committed to disk orthe like. In certain embodiments, not all the log entries are assigned atime stamp. Rather particular types of data, such as for example,consistency point markers and/or database commit entries, are assignedtime stamps.

In certain embodiments of the invention, the data agent 236 coordinateswith the replication agent 356 to copy log files from the source logs244 to the replication logs 352. Such copying may be initiated based onany suitable factor, such as, for example, preset copying intervals,capacity thresholds reached in the source logs 244, time lapsed sincethe last copy operation, replication agent 356 requests for a copyoperation, and/or based on specific parameters or requirementsassociated with a particular application 108. For instance, certaindata-sensitive applications may be copied more frequently than otherapplications in order to reduce the amount of potential data loss due toa failure occurring between copy operations.

As further illustrated in FIG. 4, the replication logs 352 include afirst log file 466 and a second log file 468. In certain embodiments,each of these log files 466, 468 corresponds, respectively, to the firstlog file 460 and the second log file 462 of the source logs 244. Forinstance, data may be transferred between the replication log(s) 352 andthe source log(s) 244 such that the order in which the data was storedin the source log(s) 244 is preserved. In addition, the log files may berecreated in the replication log(s) 352 to reflect the organization ofsource logs 244. For example, the first log file 460 and the second logfile 462 in the source logs 244 may be transferred and recreated by thereplication agent 356 and/or the data agent 236. In other embodiments,however, data may be transferred and stored in a different order withoutpreserving source system correlations and/or may be rearranged on orduring transfer to or upon arrival in replication volumes 116A, 116B.

In yet other embodiments, the transferred data may retain the datastructure used in the source logs 244 and/or may be changed, modified orrearranged to reflect, conform or otherwise interoperate with, themanagement particulars of the replication module 114. For example,certain referential indicators associated with the data may be modifiedor conformed to meet requirements of the replication agent 356. However,in preferred embodiments, referential integrity of the data ismaintained between the source system 102 and the destination system 104.

The illustrated destination system 104 further comprises an optionalpreference database 470 in communication with the replication agent 356.In certain embodiments, the preference database 470 includes storagepolicies or other preferences usable by the replication agent 356 inmanaging data. For instance, the stored preferences may indicate thedesired frequency at which the threads 358 should copy the data from thedestination logs 352 to the replication volumes 116A, 116B. Thepreference database 470 may also store path information for detailing towhich location(s) on the replication volume(s) 116A, 116B the data inthe replication log(s) 352 should be copied. In yet other embodiments,the preference database 470 may include storage policies that dictateparticular criteria for performing one or more data managementoperations on the replicated data.

With continued reference to FIG. 4, the replication module 114 furthercomprises one or more processes, such as a replication set or a logprocessing module 469 with a first thread 358A and a second thread 358B.In certain embodiments, as discussed above, the threads 358A, 358B areinstantiated by the replication agent 356 to transfer data from thefirst and second replication logs 466, 468 to the first replicationvolume 116A and/or the second replication volume 116B.

In certain embodiments, the threads 358A, 358B process, scan and/ortraverse the replication logs 352 and scan log entries therein toidentify the data or files of concern to the particular thread. Incertain embodiments, the threads 358A, 358B scan the log entries in asequential manner (e.g., in the order in which the log entries werewritten to the log file). When the thread encounters a log entry ofinterest, the thread copies the particular log entry from thereplication logs 352 to the replication volumes 116A, 116B. Forinstance, the log entries may be copied in accordance with a pathestablished based on the correlation or pairing information provided bythe data agent 236 to the replication agent 356.

In certain embodiments, the threads 358A, 358B utilize time stamp orother temporal information that enables processing and/or replaying ofmodification operations. For example, based on time stamp information,the threads 358A, 358B may rearrange the replication data such that thedata is stored on the one or more replication volumes in the properorder (e.g., the order in which the data was intended to be written tothe source storage device 112). In such embodiments, the replicated datamay be subsequently retrieved, recalled or otherwise accessed orprocessed and may be used to accurately restore the state of theapplication 108 as it existed at a given point in time. In yet otherembodiments, other data management operations (e.g., searching, dataclassification) may be performed on the replicated data.

In certain embodiments, instantiated threads 358A, 358B may operate inparallel or sequentially to scan one or more replication logs 352 forlog entries to copy for a certain application 108. Each thread, whichmay be responsible for replicating certain data of the application 108,may continue to scan the replication log 352 until encountering aconsistency point. When such occurs, the thread may then notify thereplication agent 356 that it has reached a consistency point and/orpause operation. When all active threads for a specific application 108notify the replication agent 356 that a consistency point has beenreached, the replication agent 356 may identify the data at that pointin time as representing a known good state of the application 108. Incertain embodiments, at this point, the replication agent 356 maysuspend further copy operations by the threads 358A, 358B while thereplicated data represents a known good state of the application 108. Inyet other embodiments, the replication agent 356 may monitor theoperation of the threads 358A, 358B without waiting for a notificationfrom the threads.

Generally, the disclosed systems may handle files that may be needed torestore an application type (e.g., MICROSOFT EXCHANGE) if a data failurewere to occur. This information gathering for the various files (e.g.,control files, configuration files or the like) may preserve thereferential integrity of one or more predefined application typesoperating within the storage operation system.

The replication system 400 of FIG. 4 provides several advantages forperforming copying or other storage operations to data. For example, incertain embodiments, the replication system 400 may perform storageoperations on the replicated data (e.g., data stored in the replicationvolumes 116A, 116B).

That is, creating replication copies allows the replication system 400to access copies of production volume data without the significant useof client system 102 resources and/or interrupting or suspending dataoperations to the source storage device 112, thereby reducing the impactof data management operations on client applications. In addition,consistency point (known good state) information along with time stampinformation may be advantageously used in performing storage operationsto logically associate a time stamp of the copied data with the originaltime of the consistency point entry (e.g., the time at which theapplication 108 was in a known good state). Thus, even though thestorage operation on the replicated data is performed at a later pointin time (e.g., in relation to the client system time of the known goodstate of the application), the copied data resulting from the storageoperation is associated with the original consistency point time (e.g.,the client system time of the known good state of the application)Furthermore, this logical association of the client system time of theapplication's known good state may be repeated for subsequent copies ofthe data (e.g., a backup of the snapshot of the replicated data).

In certain embodiments, instructions for the storage operations are sentfrom the data agent 236 on the source system 102. For instance, theinstructions may be included in the log file entries copied from thesource system 102. In yet other embodiments, the storage operations arecoordinated by the replication agent 356 (e.g., according to storagepolices stored in the preference database 470) in combination with, orindependent of, the data agent 236. In yet other embodiments, policiesfor storage operations may be stored in another system managementcomponent (e.g., a storage manager module).

Examples of certain storage operations performable on the replicateddata in the destination storage device 116 will now be described. Forexample, a storage operation may include a basic copy of data from afirst location to a second location.

Another form of a storage operation that may be performed on thereplicated data is a backup operation. A backup operation generallyincludes copying data into a backup format as opposed to a nativeapplication format. For example, a backup copy may be stored in a backupformat that facilitates compression and more efficient long-termstorage. Backup copies generally have relatively long retention periodsand may be stored on media with slower retrieval times than other typesof secondary copies and media. In some cases, backup copies may bestored at an offsite location.

Another form of a storage operation that may be performed on thereplicated data is a snapshot operation. In general, a snapshot recordsthe state of a storage device, file system, or volume at a certain pointin time. That is, the snapshot may be used to provide a point-in-timeimage of a live storage volume. In certain embodiments, the snapshot maybe used for backing up data and/or, in the event of failure, to restorethe storage device or volume to a previous known good state.

Snapshots may be implemented by several different methods. For example,in a “copy-on-write” method, a snapshot of a storage volume is createdusing a designated space, or snapshot volume, for the particularsnapshot. During the initial creation of the snapshot, instead ofcopying the physical data, information is recorded about the location oforiginal data in the storage volume (e.g., a logical copy of the data).For instance, metadata may be stored with respect to the original dataand/or pointers may be used to identify the location of the originaldata. In certain embodiments, this initial snapshot may be performedwhen the storage volume is quiesced or in a “frozen” state.

Successive snapshots then track changes to the storage volume as theoriginal data is modified. Before original data is modified, theoriginal data is copied to a location in the designated snapshotstorage. Thus, when a read request for an unmodified data block is madeto the snapshot volume, the request is redirected to the original copyof the data. However, if a read request is made for a data block thathas been modified, the request is directed to data in the specifiedsnapshot storage. In embodiments wherein multiple snapshots are createdand maintained concurrently, each snapshot may be designated or assigneda different storage space or snapshot volume.

In yet other embodiments, other types of snapshot technology may beused, such as, for example, direct-on-write, split mirror, copy-on-writewith background copy, continuous data protection copies, combinations ofthe same or the like. For instance, in certain embodiments, a snapshotmay not actually create another physical copy of all the data associatedwith an application, but may simply create pointers that are able to mapfiles and directories to specific disk blocks.

In certain embodiments, a snapshot of the replication data is preferablyperformed at a consistency point. The snapshot then represents a set ofrecoverable application data up to the point in time associated with theconsistency point (e.g., the time at which the quiesced applicationsuspended normal data modification operations). Thus, a snapshot copycreated from the replication data at a consistency point may also beconsistent with the data stored in the primary volume or source storagedevice 112 at the time the consistency point was generated.

In certain embodiments, a snapshot is taken for each volume in whichdata is being replicated. For instance, with reference to FIG. 4, firstthread 358A is writing to the first replication volume 116A, and secondthread 358B is writing to the second replication volume 116B. In suchembodiments, when the first and second threads 358A, 358B arrive at aconsistency point log entry, a snapshot is taken of the replicated datain each replication volume 116A, 116B.

In certain preferred embodiments, when the snapshot is performed at aparticular consistency point, the time of the snapshot is advantageouslylogically associated with the time that the consistency point wasgenerated at the client system 102 (e.g., the client system time of theknown good state of the application 108). For instance, the time stampof the consistency point may be used to logically assign a “time” to thesnapshot of the replicated data. In such a process, the snapshot of thereplicated data then appears as if the snapshot was directly taken ofthe data in the source system 102 at the time of the consistency point.Such a process allows for the snapshot data to be viewed as a directcopy of the production volume data for a particular application (e.g.,source storage device 112) at a certain point in time (e.g., the time ofa known good state of an application).

For example, consider production volume data (e.g., source system data)that represents a known good state of the application 108 as of 3:00P.M. To identify the known good state of the application, a consistencypoint marker having a time stamp of 3:00 PM is added to an appropriatelog file. At 3:05 P.M., the production volume data along with theconsistency point is replicated to a destination storage device 116.Thus, at this point in time, the destination storage device 116 is anexact copy of the production volume data of the application 108 as of3:00 P.M.

A snapshot copy of the replicated data on the destination storage deviceis then performed at 3:10 P.M. During or after performing the snapshotoperation, the snapshot is then logically associated with the 3:00 timestamp identified by the consistency point. As a result, the snapshotwill have the same or substantially the same data as if the snapshotwere directly taken from the source system data at the known good stateof the pertinent application. Moreover, through logically associatingthe 3:00 P.M. time stamp with the snapshot data, the snapshot appears tohave been directly taken from the source system data, even thought thesnapshot was taken ten minutes later. However, because the source systemdata is not actually used to perform the snapshot operation, the sourcesystem resources are not impacted and may be free for client or otheruse.

In certain embodiments, logical associations of time with respect to thesnapshot may be stored jointly with the snapshot and/or in a separatedatabase. For instance, the logical time associations may be stored inan index on one or more replication volumes, on the replication module114 (e.g., replication agent 356), in the preference database 470, in astorage manager database index, by the data agent 236, combinations ofthe same or the like. In certain embodiments, stored jointly with the“updated” time data may also be the actual location of the snapshot onthe replication volumes 116A, 116B or other storage device(s).

As discussed above, the frequency of consistency points may vary and maydepend on one or more system factors or user-selectable parameters. Forinstance, storage policies stored in one or more databases of thereplication system 400 (e.g., the preference database 470) may dictatethe frequency that consistency points are inserted (e.g., frequency ofquiescing application) and/or the frequency at which snapshots areperformed. In certain embodiments, a snapshot may be performed at eachconsistency point. In yet other embodiments, a snapshot may be performedat some other interval of consistency points (e.g., every fiveconsistency points).

Once the snapshot is performed, one or more messages to systemcomponent(s) may be sent containing information relating to the time thesnapshot was taken, the time stamp of the consistency point, thelocation of the snapshot, the location of the replication copy, theapplicable time of the copy, combinations of the same or the like. Forinstance, such a completion message may be initially sent to thereplication agent 356, which may further optionally notify the dataagent 236 or other system management components with the informationdescribed above.

While certain embodiments of storage operations have been disclosed asbeing usable with the replication system 400 of FIG. 4, a wide varietyof other storage operations may also be performed on the replicationdata and/or in conjunction with consistency point information. Forexample, other copies of the replicated data may be performed, such as,but not limited to, creation, storage, retrieval, migration, deletion,auxiliary copies, incremental copies, differential copies, HierarchicalStorage Management (“HSM”) copies, archive copies, Information LifecycleManagement (“ILM”) copies, other types of copies and versions ofelectronic data or the like.

For instance, an HSM copy is generally a copy of the primary copy data,but typically includes only a subset of the primary copy data that meetsa certain criteria and is usually stored in a format other than thenative application format. For example, an HSM copy may include onlythat data from the primary copy that is larger than a given sizethreshold or that is older than a given age threshold and may be storedin a backup format. Often, HSM data (e.g., e-mail messages) is removedfrom the primary copy, and a stub is stored in the primary copy toindicate its new location. When a user requests access to the HSM datathat has been removed or migrated, systems use the stub to locate thedata and often make recovery of the data appear transparent even thoughthe HSM data may be stored at a location different from the remainingprimary copy data.

In yet other embodiments, the replication system 400 may use thereplication data to create a shadow version of the data on the sourcestorage device 112 of the client system 102. In such embodiments, ifthere is a failure and/or overload of the source storage device 112, thesystem 102 may repoint to the shadowed version.

In yet other embodiments, different types of data management operationsmay be performed on the replication data depending on one or moreschedule policies. For example, a snapshot storage operation may bescheduled to be performed at consistency point entries every fiveminutes, while a differential backup may be performed on the replicationdata every hour. Furthermore, a full backup may be scheduled to beperformed once a day. Such scheduling policies may be advantageouslydetermined to satisfy the needs of the user while reducing the impact onsystem resources.

In certain embodiments, after appropriate storage operations areperformed on the replicated data, a message may be sent to other systemmanagement components (e.g., a snapshot manager and/or optional storagemanager) indicating that the replication process is complete up to thetime stamp associated with consistency point. At this point, thereplication agent 356 may instruct copy operations associated with thethreads 358A, 358B to resume.

FIG. 5 illustrates an exemplary embodiment of a data structure of a logentry 500 usable with the replication systems described herein. Incertain embodiments, the log entry 500 comprises information regardingmodifications to data and/or files on the source storage device 112 andmay include, for example, information regarding: which file wasmodified, the time of the modification, the type of modification, therelative data, a unique identification, combinations of the same or thelike. For exemplary purposes, the various fields of the log entry 500will be described with respect to a data write operation in thereplication system 400 of FIG. 4.

In certain embodiments, the log entry 500 is initially generated by thefilter driver 110 and is stored in the source log 244. For example, thelog entry 500 may comprise a data word having a plurality of fields. Asillustrated, the log entry 500 comprises a log entry number field 502, apath field 504, a time stamp field 506, an application type field 508, awrite type field 510, a size field 512, a checksum field 514, an offsetfield 516 and a payload field 522.

The log entry number field 502 may include information regarding theentry number assigned to the log entry 500 for system managementpurposes such that entries may be tracked and reordered relative to oneanother if necessary. For example, as mentioned herein, log entries maybe arranged in a temporally sequential manner based on the applicationwrite operation with which the particular log entry 500 is associated.In certain embodiments, log entry numbers or other information may berecycled over time once all the numbers in a particular range have beenused. In yet other embodiments, the log entry number field 502 may beconfigured to store other types of identification data for labeling thelog entry 500.

The path field 504 may include information regarding the file path onthe source storage device 112 with which the data write operation wasassociated. For example, a path of “C:\DIR\USER\” may indicate that thelog entry corresponds to an operation writing data to a folder or fileon the source storage device having the designated pathname. In certainembodiments, the path field 504 may include an absolute file pathname.In other embodiments, the path field 504 may include an abbreviatedpathname and/or an inode (e.g., for UNIX-based systems).

Moreover, the path field 504 may include information relating to the logentry's replication volume destination, and thus may be useful inestablishing or confirming correlation or pairing information used bythe thread(s) 358A, 358B. For instance, in certain embodiments, the filepath of a particular log file may be hard-coded to one or moreparticular replication volume(s).

The time stamp field 506 may include information relating to the timewhen the subject data write occurred. In certain embodiments, the timestamp is advantageously associated with the time of the client computer230 on which the application 108 is executing. For instance, the filterdriver 110 may access the source system time when generating the logentry 500. In other embodiments, the time stamp may be provided by thefilter driver 110 and/or may be relative to the replication system time.

The application type field 508 may include information identifying theapplication type with which the log entry 500 is associated (e.g.,MICROSOFT OUTLOOK data, MICROSOFT SHAREPOINT data, ORACLE data, SQLdata, MICROSOFT WORD data, MICROSOFT INTERNET EXPLORER data or thelike).

The write type field 510 may include information regarding the categoryof write data involved with the log entry 500. For instance, the writetype may identify if the log entry 500 is associated with a databasemodification, a log write, a database commit command, a consistencypoint or the like. In certain embodiments, the information in the writetype field 510 is used to implement parallelism between multiple threadswhen performing data replication. For instance, a first thread (e.g.,thread 358A) may handle log write commands, and a second thread (e.g.,thread 358B) may handle commit database commands. In certainembodiments, the data stored in the write type field 510 may be used forprioritizing the processing of various log entries (e.g., processing bythe threads 358).

The size field 512 may include information relating to the size (e.g.,the number of bytes) of the data being modified by the data writeoperation. In yet other embodiments, the size field 512 may containinformation relating to the size of other or additional segments withinthe log entry 500, such as, for example, the size of the payload field522.

The checksum field 514 may include information relating to errorchecking to ensure, for example, that the log entry 500, when createdand subsequently transmitted, contains the expected number of bits andhas not been corrupted or otherwise impermissibly changed. For instance,the checksum field 514 may store data representing the arithmetic sum ofsome or all of the fields in the log entry 500.

The offset field 516 may include information relating to the locationwithin a file or portion of data that the data write is occurring. Forinstance, if the subject data write operation is associated withmodifying the twentieth through the thirtieth bytes of a file or pieceof data fifty bytes long, the offset field 516 may store a value oftwenty. In such embodiments, the information in the offset field 516 maybe used jointly with the information in the size field 512 to identifythe entire portion of a file being modified. For instance, in the aboveexample the size field 512 may store a value of eleven to indicate thelength of the modified section (i.e., twentieth through thirtiethbytes).

The payload field 522 may include information relating to the datawritten from the application 108 to the source storage device 112. Thisinformation generally represents the application data captured by thefilter driver 110 for replication and may include additional informationfor the ongoing operation or reconstitution of the application 108.

It will be understood that the illustrative filter driver log entry 500shown in FIG. 5 merely represents one possible embodiment of a log entrysuitable for use with embodiments of the invention and that otherembodiments may be used if desired. For example, in other embodiments,the log entry 500 may comprise more or fewer fields to accommodate therequirements of the particular replication or storage operation systeminvolved and/or to achieve certain data or management goals, such asconserving memory, increasing processing speed and increasing the amountof information in each log entry. For instance, in certain embodimentswherein the path determination for a particular log file or log entry isdynamic, the log entry 500 may not include the path field 504. In otherembodiments, the log entry 500 may include a priority field that may beused for prioritizing replication and/or data management operations ofdata associated with the log entry 500.

In other embodiments, the log entry 500 may concern a file attributechange rather than a data write operation. In such embodiments, thewrite type field 510 may identify the log entry 500 as being associatedwith a file attribute change. Furthermore, the log entry 500 may storeinformation regarding the new file attribute but would not requireoffset or size values to be stored in the size field 512 and/or theoffset field 516.

FIG. 6 illustrates another embodiment of a replication system 600similar to the replication system 400 of FIG. 4. As shown, thereplication system 600 further includes a storage manager 680 thatcommunicates with the source system 102 and the replication system 104.In certain embodiments, the storage manager 680 is a software module orapplication that is configured to direct the performance of one or morestorage operations and, in particular, the replication of data from thesource system 102 to the replication system 104. In further embodiments,the storage manager 680 may perform one or more of the operations orfunctions described above with respect to the data agent 236 and/or thereplication agent 356. For instance, the storage manager 680 may directand/or coordinate the performance of one or more storage operations onthe replicated data (e.g., snapshots of the replicated data).

In certain embodiments, the storage manager 680 maintains an index 682,such as a cache, for storing information relating to: logicalrelationships and associations between components of the replicationsystem 600, user preferences, management tasks, and/or other usefuldata. For example, the storage manager 680 may use its index 682 totrack the location and timestamps of one or more snapshots of thereplicated data. In certain embodiments, the storage manager 680 maytrack logical associations between one or more media agents (not shown)and/or storage devices.

The storage manager 680 may also use its index 682 to track the statusof data management operations to be performed, storage patternsassociated with the system components such as media use, storage growth,network bandwidth, Service Level Agreement (“SLA”) compliance levels,data protection levels, storage policy information, storage criteriaassociated with user preferences, retention criteria, storage operationpreferences, and other storage-related information. The index 682 maytypically reside on the storage manager's hard disk and/or otherdatabase.

As shown in FIG. 6, the storage manager 680 further communicates with adatabase 684. In certain embodiments, the storage manager database 684comprises a memory for storing system management information relating tothe replication of data. For instance, the database 684 may beconfigured to store storage and/or restore policies, user preferences,the status or location of system components or data, combinations of thesame and the like. In yet other embodiments, the database 684 may beconfigured to store information described above with respect to theindex 682. In yet other embodiments, at least a portion of the index 682may be stored on the database 684.

In other embodiments, the storage manager 680 may alert the user orsystem when a particular resource of the replication system 600 isunavailable or congested or when components are unavailable due tohardware failure, software problems, or other reasons. In certainembodiments, the storage manager 680 may utilize replication system 600data to suggest solutions to such problems when they occur or evenbefore they occur. For example, the storage manager 680 might alert theuser that a storage device in the replication system 600 was full orotherwise congested, and then suggest, based on job and data storageinformation contained in its index cache, an alternate storage device.In yet further embodiments, the storage manager 680 or other systemcomponent may take action to remedy the problem at issue. For example,the storage manager 680 may perform load balancing, error correction, orthe like, based on information received regarding the replication system600.

In certain embodiments, the storage manager 680 may include othercomponents and/or modules. For example, the storage manager 680 mayinclude a jobs agent module (not shown) that monitors the status ofstorage operations that have been performed, that are being performed,or that are scheduled to be performed in the replication system 600.

Moreover, the storage manager 680 may include an interface agent module(not shown). In certain embodiments, the interface agent module mayprovide presentation logic, such as a graphical user interface (“GUI”),an application program interface (“API”), or other interface by whichusers and system processes may be able to retrieve information about thestatus of storage operations and issue instructions to the replicationsystem 600 regarding the performance of storage operations. For example,a user may modify the schedule of a number of pending snapshot copies orother types of copies. As another example, a user may use the GUI toview the status of all storage operations currently pending in thereplication system 600 or the status of particular components in thereplication system 600.

Additional details of storage manager modules useful with embodiments ofthe replication systems described herein are described in U.S. patentapplication Ser. No. 09/354,063, filed Jul. 15, 1999, which is herebyincorporated herein by reference in its entirety.

FIG. 7 illustrates a simplified flowchart of an initialization process700 in accordance with certain embodiments of the invention. Inparticular, the initialization process 700 concerns certain preliminaryprocesses and acts for setting up a system for performing datareplication, as disclosed herein. For exemplary purposes, theinitialization process 700 will be described hereinafter with referenceto the components of the replication system 400 of FIG. 4.

The initialization process 700 begins with Block 705, wherein one ormore data agent(s) 236 are installed on the client computer 230. Incertain embodiments, the data agent 236 may be installed remotely fromother portions of the replication system 400 based on a particular needor to conform to certain directives or resident storage policies. Inother embodiments, the data agent 236 may be installed locally by asystem user as desired. For instance, installation of the data agent 236may include deployment and installation of object code files andsupporting software.

In certain embodiments, the data agent 236 may be installed for eachapplication 108 of interest, or one or more data agents 236 may beinstalled for a larger number of applications 108. Furthermore, incertain embodiments, an installation guide such as a wizard or otherprogram may recommend the appropriate number and type of data agents 236to install (which may be performed substantially automatically based onapplication and system configuration information).

At Block 710, the installed data agents 236 may perform certainauto-discovery routines in order to determine basic system andapplication information. In some embodiments, the auto-discoveryroutines may be considered part of the installation process. Forexample, the data agent 236 may begin the auto-discovery process byscanning and evaluating the folder and directory structure of the clientcomputer 230 to determine which folders are used by a particularapplication 108. In certain embodiments, such information allows thedata agent 236 to identify and locate files or other informationnecessary to replicate the current operating state of the application108 of interest.

In certain embodiments, the scanning and evaluation process may involvescanning multiple physical and/or logical volumes associated with thesource storage device 112 and/or within a given network or enterprise tolocate the data and system configuration information necessary for datareplication.

After the appropriate resources have been discovered and examined, thedata agent 236 may identify, arrange, coordinate and/or queue thenecessary data within various locations or instances of the application108 to establish a platform for proper data replication (Block 715). Incertain embodiments, this process may be a precursor for performing theinitial seeding operation described above.

Next, at Block 720, the data agent 236 communicates with the replicationagent 356. For instance, the data agent 236 may transmit to thereplication agent 356 information regarding the replication of data. Thedata agent 236 may also request information from the replication agent356 and/or other network management components for any information thatmay bear on, or be related to, the correlation or mapping of networkstorage paths for replication data. For example, the data agent 236 mayconsult the preference database 470 of the destination system 104, thepreference database 465 of the source system 102 and/or a storagemanager component, for correlation or pairing information. Based on thisinformation, data paths may be identified for use by threads 358 whencopying data from the replication logs 352 to the replication volumes116A, 116B. In certain embodiments, one or more data paths may bedynamically coded or determined, such as, for example, based on one ormore storage policies and/or preferences.

At Block 730, the initialization process 700 includes installing andinitializing the filter drivers 110. In certain embodiments, suchinstallation and/or initialization is based at least in part oninformation obtained by the data agent 236 during the discovery process(Block 710). For example, in certain embodiments, one or more filterdrivers 110 may be installed by the data agent 236 in the I/O path ofthe application(s) 108.

FIG. 8 illustrates a simplified flowchart of a replication process 800in accordance with certain embodiments of the invention. In particular,the replication process 800 involves the copying of data from a sourcesystem to a destination system. Furthermore, in certain embodiments, thereplication process 800 is configured to be performed after completionof the initialization process 700 of FIG. 7. For exemplary purposes, thereplication process 800 will be described hereinafter with reference tothe components of the replication system 400 of FIG. 4.

The replication process 800 begins with Block 805, wherein the filterdriver 110 populates the source log(s) 244 with data associated with theapplication 108, such as data identified by the data agent 236. Asdiscussed in more detail above, such data may relate to data or filemodification operations being passed from the application 108 to thesource storage device 112. In certain embodiments, the filter driver 110populates the source logs 244 in a temporally sequential manner suchthat operations and data are recorded in time descending (or ascending)order (e.g., first operation at the top and last operation at thebottom).

In certain embodiments, the data is populated in the source logs 244 ina format similar to the structure of the log entry 500 of FIG. 5. Inother embodiments, the data may be populated in other suitable formatsto satisfy the requirements of the particular replication system. Forinstance, the log file format may comprise a two- or multi-columnstructure, wherein the information in a first column may indicate thetype of data operation performed, and the log entry's position in thelog file indicates the order of the operation relative to otheroperations in the log file. The information in a second column mayindicate the payload data associated with the data operation indicatedby the first column.

After or concurrently with Block 805, the data agent 236 or other systemcomponent pauses or quiesces the application 108 (Block 810). Asdiscussed above, such quiescing causes the application 108 totemporarily suspend data modification operations to the source storagedevice 112 once the application 108 reaches a known good state.

Once new modification operations are suspended and the associated sourcelog 244 is populated based on the modification operations up to theknown good state, the data agent 236 or other replication systemcomponent inserts a logical marker or tag in the source log 244 (Block815). This “consistency point” denotes that the state of the data issuch that the application 108 may be recovered or that further stableoperation from that point going forward is ensured. Once the consistencypoint is identified and established, the data agent 236 may restart theapplication 108 such that data modification operations from theapplication 108 to the source storage device 112 resume.

As referenced by Block 820, the data agent 236 or other systemmanagement component coordinates the transfer of the data in the sourcelogs 244. In certain embodiments, the data agent 236 coordinates withthe replication agent 356 to copy data from the source logs 244 to thereplication log(s) 352. For instance, the replication agent 356 and/ordata agent 236 may open a network path or a communication socket betweenthe source log(s) 244 and the replication log(s) 352. The log entries ofthe source log(s) 244 may then be transferred as described above topopulate the replication log(s) 352. In certain embodiments, as thereplication log 352 is populated, the replication agent 356 may alsoobtain configuration information from the data agent 236 or other systemmanagement component such as, for example, a storage manager. Suchconfiguration information may identify aspects of the set of informationbeing transferred as well as identify pairing information thatcorrelates certain types of replication data with certain replicationvolumes or other storage destinations.

At Block 825, the replication process 800 includes instantiating one ormore threads 358 to begin the transfer of data from the replicationlog(s) 352 to certain replication volumes 116A, 116B. In certainembodiments, the replication agent 356 is configured to instantiate oneor more of the threads 358A, 358B. In certain embodiments, the threads358 are instantiated and/or particularized based on pairing orcorrelation information received from a management component and/orbased on certain system configuration information (e.g., availablereplication volumes), data path information, the type of information inthe transferred data set, combinations of the same and the like. Forexample, the replication agent 356 may instantiate one or more threads358 that correlate certain data types with certain data volumes and mayspecify primary and alternate data paths.

Once instantiated, the threads 358 process and/or traverse thereplication log(s) 352 until a consistency point is encountered (Block830). In certain embodiments, when reaching a consistency point, thethread 358 stops scanning the replication log 352 and notifies thereplication agent 356 that the thread 358 has reached the consistencypoint (Block 835).

In certain embodiments, once all active threads 358 associated withtraversing the replication logs 352 have notified the replication agent356 that a consistency point has been reached, the replication process800 moves to Block 840. At this point, the replicated data stored in thereplication volumes 116A, 116B preferably represents a known good stateof the application 108.

At Block 840, the replication agent 356 suspends further operation bythe threads 358. For instance, the replication agent 356 may suspenddata writes to the destination volumes 116A, 116B. At this point, thereplication process 800 proceeds with Block 845, wherein one or morestorage operations (e.g., snapshots) may be performed on the replicateddata, which are described in more detail above.

FIG. 9 depicts a block diagram of another embodiment of a replicationsystem 900, which has multiple replication modules. In certainembodiments the replication system 900 advantageously provides loadbalancing options between the replication modules and/or the parallelprocessing of replication data.

As shown, the replication system 900 comprises the source system 102that communicates through the network 106 with a plurality ofreplication modules 114 a, 114 b and 114 c. For simplicity, the varioussub-components of the source system 102 and the replication modules 114a-114 c will not be described in detail, and it will be understood thatsuch sub-components, and the functions thereof, may be similar to thosedescribed with respect to FIGS. 1-4.

In certain embodiments, each of the replication modules 114 a-114 c isconfigured to receive a portion of replication data from the sourcesystem 102. For example, one or more data agents of the source system102 may communicate with replication agents of the replication modules114 a-114 c to coordinate the transfer data from source logs toreplication logs. In certain embodiments, one data agent may communicatewith one or more replication modules, or multiple data agents maycommunicate with the same replication module.

For instance, in certain embodiments, each replication module may beconfigured to receive a particular type of application data from thesource system 102. As an example, replication module 114 a may beconfigured to receive replication data related to MICROSOFT EXCHANGE,while replication modules 114 b, 114 c are configured to receivereplication data relating to, respectively, SHAREPOINT and SQL data. Inyet other embodiments, each replication module may handle differenttypes of data from the same application.

In other embodiments, the replication data may be sent to each of thereplication modules 114 a-114 c based at least in part on the workingload of the replication modules 114 a-114 c. In such embodiments, thereplication system 900 may further comprise a storage manager or othersystem component (not shown) that may monitor the state of each of thereplication modules 114 a-114 c. In yet other embodiments, thereplication modules 114 a-114 c may communicate a status report to thedata agent(s) of the source system 102. When one of the replicationmodules 114 a-114 c has an unbalanced load compared to the otherreplication modules, the replication data from the source system 102 maybe re-routed or directed to one or more of the other replicationmodules.

It will also be understood that, in certain embodiments, the replicationmodules 114 a-114 c may also communicate with each other either directlyor through the network 106. For example, the replication modules 114a-114 c may share with each other information regarding: storagepolicies, storage location information, processing loads, storagecapacities, combinations of the same or the like.

As further depicted in FIG. 9, each of the replication modules 114 a-114c is associated with a destination storage device, the functionality andstructure of which is described in more detail above. In certainembodiments, each of the destination storage devices 116 a-116 c is aseparate device. For instance, one or more of the destination storagedevices 114 a-114 c may be remotely located with respect to the otherdestination storage devices. In yet other embodiments, one or more ofthe destination storage devices 114 a-114 c may reside on the samephysical medium, such as separate volumes on the same physical media.

In certain embodiments, the replication system 900 further comprises asecond tier having one or more destination storage devices 116 d thatare used to store copy data generated from storage operations performedon the replication data stored in the destination storage devices 116a-116 c. For instance, as shown, in communication with the destinationstorage devices 116 a-116 c is a media agent 902. In certainembodiments, the media agent 902 comprises a module (software and/orhardware) that conducts, copies or otherwise moves data between thedestination storage devices 116 a-116 c and the destination storagedevice 116 d. The media agent 902 may generally communicate with thedestination storage devices 116 a-116 d via a local bus, such as a SCSIadaptor. In other embodiments, the destination storage devices 116 a-116d may be communicatively coupled to the media agent 902 via a SAN and/ora NAS.

In certain embodiments, the media agent 902 may maintain an index cachethat stores data generated by the replication system 900 during storageoperations. Such data may include, for example, information regardingthe location of the stored data on the destination storage devices 116a-116 d, information regarding the content of the data stored, such asfile names, sizes, creation dates, formats, application types, and otherfile-related criteria, information regarding the client system 102and/or the replication modules 114 a-114 c.

In certain embodiments, the media agent 902 and/or a storage manager maystore information regarding one or more storage policies, storagecriteria, or storage preferences associated with the stored replicateddata. For example, in certain embodiments, the media agent 902 isconfigured to obtain partial snapshots of data from multiple destinationstorage devices 116 a-116 c and merge the partial snapshots into asingle snapshot of data for storage on the destination storage device116 d.

For instance, in certain embodiments, the media agent 902 may obtainpartial snapshots that are logically associated with the same timestamp. That is, partial snapshots of replication data that are generatedwith respect to the same consistency point may be forwarded to the mediaagent 902 and constructed into a single snapshot of the data. Becausethe time stamp logically associated with the merged snapshot refers to aknown good state of the application at the time a consistency point wasgenerated at the client system, it does not matter the time(s) at whichthe partial snapshots of the replicated data were actually taken and/ormerged. Thus, the media agent 902 may be capable of merging snapshotsthat were taken at different times but that refer to the same known goodstate of the application.

Furthermore, in certain embodiments, the media agent 902 or other systemcomponent may logically associate the newly merged snapshot with thetimestamp of the relevant consistency point. Thus, multiple storageoperations may be repeatedly performed on the replication data withoutlosing referential integrity with the production volume data stored onthe client system.

While performing data copies, computer networks are sometimes faced withnetwork errors and/or rebooting. For instance, a network connection maybe lost between two sites and/or transmitted data may be corrupted. Inview of the foregoing, it would be advantageous to have a replicationsystem that further includes a process for verifying that log entriesare received and/or properly processed or replayed once on thedestination machine(s).

FIG. 10 illustrates an embodiment of a rolling cache configuration 1000that provides for the tracking of the last successfully applied recordon the destination machine(s), preserving not-yet-transferred recordseven across reboots, and/or switching back to an initial synchronizationmode if too much time has passed during a network failure. For instance,in certain embodiments, the rolling cache configuration 1000 performscaching of log entries in the application layer once the log entrieshave been received from a filter driver.

In certain embodiments, the rolling cache configuration 1000 maintains arolling cache 1002 of log entries on a source system and keeps track ofthe last successfully applied sequence number on each of thedestinations. In general, the rolling cache 1002 stores log entriesgenerated by the source system, which log entries are subsequentlyretrieved by one or more replication threads (or processes). In aUNIX-based environment, the log entries in the rolling cache 1002 havepreferably been assigned a destination file pathname (such as throughpathname translation), which is described in more detail below withrespect to FIGS. 11-13.

In certain embodiments, the rolling cache 1002 comprises a permanentdisk cache whose storage that is not affected by system reboots or thelike. In certain embodiments, the rolling cache 1002 is configured tomaintain its storage space by performing in a first-in first-outfashion. In particular, when new log entries are added to the rollingcache 1002, the oldest log entries are deleted. Such embodimentsadvantageously reduce fragmentation and the need for disk cleanups ofthe rolling cache 1002.

In certain embodiments, the size of the rolling cache 1002 is selectableby a user. For instance, the user may input through a user interface theamount of memory on a disk to be allocated for the rolling cache 1002.In other embodiments, the size of the rolling cache 1002 is fixed and/orpredetermined. For example, in certain embodiments, the rolling cache1002 maintains a log entry history of between approximately fivegigabytes and approximately twenty gigabytes. In other embodiments, therolling cache 1002 has a smaller size or a larger size to meet the needsof the particular replication system (e.g., depending on the size ofindividual log entries).

As depicted in FIG. 10, the rolling cache 1002 further communicates witha plurality of replication threads: a first replication thread 1010, asecond replication thread 1012, a third replication thread 1014 and afourth replication thread 1016. In certain embodiments, each of thereplication threads 1010, 1012, 1014, 1016 communicates through anetwork 1020 with one or more destination systems. For example, thefirst replication thread 1010 may communicate with a first destinationsystem 1030, the second replication thread 1012 may communicate with asecond destination system 1032, the third replication thread 1014 maycommunicate with a third destination system 1034, and the fourthreplication thread 1016 may communicate with a fourth destination system1060. In other embodiments, multiple replication threads may communicatewith a single destination system, and/or a single replication thread maycommunicate with multiple destination systems.

In certain embodiments, the replication threads 1010, 1012, 1014, 1016obtain log entries from the rolling cache 1002 and forward the entriesto the appropriate destination system(s) for data replication. Because ahistory of the log entries is preferably stored in the rolling cache1002, the replication threads 1010, 1012, 1014, 1016 do not need tostore copies of the transmitted log entries, even if the particulardestination system happens to be offline. In other embodiments, thereplication threads 1010, 1012, 1014, 1016 monitor which log entrieshave been successfully replayed on the destination system(s).

For instance, in certain embodiments, each of the log entries comprisesa unique identification, such as, for example, a sequence number. When aparticular log entry is replayed on a destination system, the subjectdestination system records the sequence number of the most recentlystored log entry. In certain embodiments, when one of the replicationthreads establishes, or reestablishes, a network connection with one ofthe destination systems 1030, 1032, 1034, 1036 (e.g., after a networkglitch or reboot), the replication thread requests from the destinationsystem the sequence number of the most recently (and successfully)replayed log entry. The replication thread then uses the sequence numberto retrieve the next log entry from the rolling cache 1002, and thereplication process resumes from where the process was previouslysuspended or aborted. In circumstances when too much time has passedsince the network failure, and the rolling cache 1002 has deleted thelog entries at issue (e.g., “rolled over”), then the replication systemmay fall back to an initial mirroring in order to resume normaloperation.

In other embodiments, the destination system(s) send the sequence numberof the most recently replayed log entry to the particular destinationthread that transmitted the log entry. In other embodiments, thereplication system may store the sequence number of the most recentlyplayed log entry. In yet other embodiments, the replication system maynotify all the replication threads 1010, 1012, 1014, 1016 and/or otherdestination systems that a particular log entry has been successfullyreplayed.

In certain embodiments, because the rolling cache 1002 provides asubstantial history of all logged changes, the replication system iscapable of recovering from network glitches or destination machinereboots without having to resynchronize the entire system. That is, thereplication thread(s) are able to find the appropriate log entry (e.g.,the log entry following the latest successfully replayed log entry) inthe rolling cache 1002 and resume replication traffic to the destinationsystem.

Embodiments of the rolling cache configuration 1000 of FIG. 10 mayprovide several advantages. For example, in certain embodiments, thereplication systems 1030, 1032, 1034, 1036 do not need to sendacknowledgments to the replication thread(s) that a particular log entryhas been received by the replication system(s). In such embodiments, thereplication threads 1010, 1012, 1014, 1016 are configured to stream ortransmit log entries to the destination systems without waiting for aresponse, thus resulting in a more rapid transmission of data.Furthermore, components of the source system need not store informationregarding which log entries have been replayed.

Although the rolling cache configuration 1000 of FIG. 10 illustrates aplurality of replication threads 1010, 1012, 1014, 1016 and destinationsystems 1030, 1032, 1034, 1036, in other embodiments, the rolling cacheconfiguration 1000 may comprise more or fewer replication threads and/ordestination systems. Furthermore, in certain embodiments of theinvention, the rolling cache configuration 100 may comprise morereplication threads than destination systems or more destination systemsthan replication threads.

As discussed above, one of the advantages of the embodiments of the datareplication systems disclosed herein is that such systems are capable oftranslating information intercepted by a filter driver on a first(source) system into information that is suitable for replay (e.g.,replication) on a second (destination) system. In certain embodiments,however, the identification of files or directories in the source systemmay not be suitable for use with the directory structure of thedestination system.

For example, in UNIX-based systems, such as SOLARIS and LINUX, filesystem operations are generally identified as operations on “inodes” (or“vnodes”) such that files are referenced by a unique inode number and/orby a combination of one or more directory inode numbers and a shortname. Such systems often utilize pathname translation algorithms toimplement a user-level hierarchical view of the file system.

Such usage of inodes and short names, however, is not conducive forreplaying data modification operations on a second system, such asoccurs in the data replication systems disclosed herein. That is, a pathhaving one or more inodes and/or short names does not provide adestination system with the appropriate information for performing thereplicated data modification operation.

Certain operating systems (e.g., SOLARIS 10, LINUX 2.6) perform pathnametranslation within the operating system kernel by generally traversingbackwards a directory name lookup cache (DNLC). Using such translationsystems in the data replication environment, however, may yieldconcurrency issues if certain locking processes are not performed. Forinstance, in order to ensure that other threads or processes do notrename one of the components of a file's absolute path between the timethat the thread computes the absolute path and the time that a relevantlog entry is emitted, the DNLC would need to be locked against updatesfrom other threads during that period of time. Having this central lockon the DNLC, however, may impose severe performance penalties on theentire operating system.

FIG. 11 illustrates a block diagram of an exemplary embodiment of apathname translation system 1100 that is configured to more efficientlyperform pathname translation in a data replication system. For example,in certain embodiments, the translation system 1100 is advantageouslyconfigured to convert inode numbers (such as those used inside thekernel driver and/or associated virtual file system handlers) of asource system into absolute file pathnames to be used on one or morereplication systems. In certain embodiments, all or a portion of thepathname translation is advantageously implemented in the applicationspace external to the kernel space (e.g., in “userland”), therebyreducing potential loads on the source system.

As shown, the pathname translation system 1100 comprises a filter driver1110. In certain embodiments, the filter driver 1110 is configured tomonitor data management operations, such as data write operations orfile attribute modification operations, associated with a computerapplication executing on a source computer. For instance, suchoperations may comprise changes to data in a production level memory.Examples of embodiments of filter drivers usable with the pathnametranslation system 1100 are described in more detail herein.

The filter driver 1110 is further configured to populate a queue 1112with log entries, or “raw” journal entries, related to detected datamodification operations from the application. In certain embodiments,the log entries generated by the filter driver 1110 are each associatedwith an inode that identifies to which directory and/or file on thesource storage device the associated data modification was directed. Thequeue 1112 is configured to store the log entries until they areprocessed by a driver thread (or process) 1114. In certain embodiments,the queue 1112 is implemented in volatile memory on the source system.

The queue 1112 forwards the log entries to the driver thread 1114. Incertain embodiments, the driver thread 1114 polls the queue 1112 fornewly-generated log entries by the filter 1110. The driver thread 1114subsequently stores the log entries in a buffer 1116. In certainembodiments, the buffer 1116 may be labeled a “raw” buffer in that it isconfigured to store “raw” log entries, which were generated by thefilter driver 1110 and/or which do not yet have an absolute filepathname.

In certain embodiments, the buffer 1116 is a memory-based queue forstoring the log entries until processed by a database thread (orprocess) 1118. In certain embodiments, the buffer 1116 advantageouslyfacilitates and/or expedites the unloading of raw records from expensivedriver memory to swappable application memory. For instance, the buffer1116 may comprise an application level-buffer of a size betweenapproximately 40 megabytes and approximately 60 megabytes. In certainembodiments, the buffer 1116 is advantageously implemented as a first-infirst-out buffer.

In certain embodiments, the database thread 1118 is advantageouslycapable of performing inode-to-pathname translation for each of the logentries in the buffer 1116. After performing the translation, thedatabase thread 1118 may send the log entry (with the absolute filepathname instead of the inode entry) to a desired destination, such as areplication system, for further processing. In yet other embodiments,the database thread 1118 sends the log entry to a cache, such as therolling cache 1002 of FIG. 10, on the source system prior to the logentry being sent to a replication system.

In certain embodiments, the database thread 1118 is configured to accessa pathname database 1120 to enable the thread 1118 to perform pathnametranslation. The pathname database 1120 advantageously storesinformation that associates one or more inodes or short names with anabsolute file pathname. In yet other embodiments, the pathname database1120 may comprise other means or data for performing pathnametranslation, including, but not limited to, a flat table, customizedcode, combinations of the same or the like.

In certain embodiments, the database thread 1118 is configured tosequentially process and perform pathname translation for each of thelog entries and/or perform database updates. In certain embodiments,because the entries are processed in the same order as recorded by thesource storage system, the consistency of storage file names inpreserved in the pathname database 1120 each time a new record arrives.In certain further embodiments, a particular file system is associatedwith only a single database thread 1118. In certain embodiments, use ofa single thread to perform pathname translation also facilitatessynchronization between the pathname database 1120 and the source system(e.g., source file system). Because all the database lookups and changesto the pathname database are being performed by or through the databasethread 1118 in a serialized manner, unwanted modifications to pathnamesprior to accessing the pathname database 1120 are avoided.

In certain embodiments of the invention, accessing the pathname database1120 introduces delay into the translation system 1100. For example, atcertain points in the replication process, the filter driver 1110 maygenerate log entries at a quicker pace than the pathname translationsbeing performed by the database thread 1118. For instance, high activitydisk lookups in the database 1120 for each log entry may require moretime than the generation of the log entries by the filter driver 1110.

In such embodiments, the buffer 1116 is advantageously capable ofadapting itself to the speed of the database thread 1118. For example,when the lookups by the database thread 1118 are relatively fast, thebuffer 1116 does not introduce significant delay into the data flow(e.g., relatively no performance degradation due to the buffer 1116).Thus, the buffer 1116 may be advantageously sized to be relativelytransparent to the data stream (e.g., has a small footprint). However,when the database lookups begin to slow down, the buffer 1116 is able tostore multiple log entries until the database thread 1118 is able tocatch up.

In certain embodiments, the database lookups by the database thread 1118may become so time intensive that the maximum storage capacity of thebuffer 1116 is reached. In such embodiments, the buffer 1116 isconfigured to provide disk swapping functionality to avoid overflow ofthe buffer 1116, which may result in memory problems and/or abortingreplication. For instance, as shown in FIG. 11, the buffer 1116 maystore excess log entries in a folder in memory 1122. In certainembodiments, the memory 1122 may comprise a disk and/or may be locatedon the storage device of the source machine.

In performing the swapping, the buffer 1116 forwards the most recent logentries to the memory 1122. As the database thread 1118 retrieves andprocesses the least recent log entries in the buffer 1116, the buffer1116 retrieves the log entries stored in the memory 1122.

Disk swapping, as discussed above, may also introduce delay orperformance degradation into the translation system 1100. Thus, incertain embodiments, the translation system 1100 is configured tomonitor the status of the queue 1112 and/or the buffer 1116 to determinewhether the accumulation of non-processed log entries exceeds aparticular threshold. For instance, such monitoring of the queue 1112may be performed by the filter driver 1110 and/or other managementcomponent external to the translation system 1100. In certainembodiments, the filter driver 1110 utilizes a counter to track thenumber of log entries stored in the queue 1112.

In certain embodiments, if the accumulation of non-processed log entriesbecomes too high, the translation system 1100 is configured to throttlethe application and/or other applications running on the system. Forexample, the filter driver 1110 may monitor the memory required by thestored log entries, and when the memory exceeds a given threshold,throttling of the application takes place. For instance, a thresholdbetween approximately 30 megabytes and approximately 60 megabytes may beestablished such that application throttling occurs when the thresholdis exceeded.

In certain embodiments, application throttling includes the filterdriver 1110 introducing delays into the input/output path of theapplication, such as by holding on to data modification operationsrather than letting them pass through to the file system. For instance,the filter driver 1110 may cache the data modification operations untilthe database thread 1118 has processed the excess log entries. In yetother embodiments, the filter driver 1110 may issue a command to suspendthe user application (e.g., for a period of a few milliseconds), suchas, for example, a particular operation of the application.

In certain embodiments, the driver throttling of the application isindependent of the disk swapping processes of the buffer 1116. In yetother embodiments, communication may exist between components of thetranslation system 1100 such that the driver throttling process is usedin conjunction with disk swapping.

Although the translation system 1100 has been described with referenceto particular embodiments, other embodiments of the translation system1100 may take on other configurations and/or functions. For example, incertain embodiments, the translation system 1100 may function withoutthe queue 1112 and/or the driver thread 1114. In such embodiments, thebuffer 1116 may be implemented in the filter driver 1110 toadvantageously reduce the number of copies made of the log entries. Sucha configuration, however, may utilize additional memory of the operatingsystem kernel.

FIG. 12 illustrates an embodiment of a pathname database 1120 of thetranslation system 1100 of FIG. 11. In particular, the pathname database1120 may be advantageously accessed by the database thread 1118 whendetermining an absolute file pathname for one or more log entries.

The illustrated pathname database 1120 is configured forinode-to-pathname translation, such as for a UNIX-based system. Inparticular, the pathname database 1120 includes three columns: adirectory inode (or parent inode) column 1222, a short name column 1224and an entry inode column 1226.

In certain embodiments, each inode in a UNIX-based system is recorded asan entry in the pathname database 1120. For instance, FIG. 12illustrates a system having four inodes, each having a single entry inthe entry inode column 1226 and having a value of “1” through “4. ” Thecorresponding short name column 1224 identifies the short name of thefile or folder associated with the particular inode. For instance, entryinode “4” identifies a folder or file with the short name of “user,”while entry inode “1” identifies a root directory. The directory inodecolumn 1222, or parent inode column, identifies the inode of the parentdirectory to the particular entry inode. For instance, entry inode “3,”which has a short name of “file,” is a child of the folder with an inodeof “2.”

As can be seen from the illustrated pathname database 1120, when thedatabase thread 1118 receives a log entry with a particular inode, thedatabase thread 1118 is able to access the pathname database 1120 andconstruct an absolute file pathname using the information storedtherein.

FIG. 13 illustrates an embodiment of a simplified pathname translationprocess 1300, such as may be performed by the database thread 1118 ofFIG. 11 in conjunction with the pathname database 1120 of FIG. 12. Forexample, the pathname translation process 1300 may be used to translatean inode to a pathname, such as an absolute file pathname to be used bya destination system in replicating data.

As shown, the translation process 1300 begins at Block 1305, wherein thedatabase thread 1118 receives a log entry to be processed. For example,with reference to FIG. 11, the database thread 1118 may retrieve the logentry from a buffer 1116. In certain embodiments, the log entrypreferably represents a data modification operation associated with aparticular application on the source system.

At Block 1310, the database thread 1118 identifies the inode associatedwith the particular operation represented by the log entry. Forinstance, the inode may represent a file or folder to which data is tobe written. In other embodiments, the inode in the log entry mayidentify a file name to be modified or other data or file modificationoperation.

At Block 1315, the database thread 1118 accesses the pathname database1120 to acquire information for translating the inode to an absolutefile pathname. In particular, the database thread 1118 searches theentry inode column 1226 for an entry that corresponds to the value ofthe log entry inode. Once the corresponding inode entry is found, thedatabase thread 1118 determines (and stores) the associated short namefrom the short name column 1224 (Block 1320).

The translation process then proceeds with Block 1325. If the subjectinode does not correspond to the root directory (“/”), the databasethread 1118 identifies from the directory inode 1222 the inode of theparent directory (Block 1330). The database thread 1118 then searchesthe entry inode column 1226 for the parent directory inode (Block 1335)and adds the short name associated with the parent directory inode tothe absolute file pathname (Block 1340).

The translation process 1300 then returns to Block 1325 to repeat thelookups and construction of the absolute file pathname until thedatabase thread 1118 reaches the root directory. Once the databasethread 1118 reaches the root directory, the database thread 1118 storesthe fully translated file pathname with the associated log entry (Block1345), and the translation process 1300 terminates.

For exemplary purposes, the translation process 1300 will be now bedescribed with reference to a data write command “vop-write (4, DATA)”and the values illustrated in the pathname database of FIG. 12. To beginthe translation process, the database thread 1118 receives the log entryrepresenting the command “vop-write (4, DATA)” (Block 1305) whichcorresponds to writing “DATA” to inode “4” on the source system (Block1310).

The database thread 1118 then accesses the pathname database 1120 andsearches the entry inode column 1226 for a value of “4” (Block 1315).Upon finding “4” in the entry inode column 1226, the database thread1118 determines from the short name column 1224 that the short namecorresponding to inode “4” is “user” (Block 1320).

Because inode “4” does not correspond to the root directory (Block1325), the database thread 1118 identifies from the directory inodecolumn 1222 that the parent directory inode of inode “4” is inode “2”(Block 1330). The database thread 1118 then returns to search the inodeentry column 1226 for the inode value of “2” (Block 1335), determinesthat the short name for inode “2” is “dir,” and adds “dir” to the filepathname (Block 1340).

Because inode “2” does not correspond to the root directory (Block1325), the database thread 1118 identifies from the directory inodecolumn 1222 that the parent directory inode of inode “2” is inode “1”(Block 1330). The database thread 1118 then searches the inode entrycolumn 1226 for the inode value of “1” (Block 1335) and determines thatthe inode “1” corresponds to the root directory (“/”) (Block 1340).

Now that the database thread 1118 has encountered the root directory(Block 1325), the database thread 1118 stores the translated filepathname (i.e., “/dir/user”) with the subject log entry, and thetranslation process 1300 terminates.

It will be understood that the translation process 1300 may differ inother embodiments of the invention in order to suit the needs of theparticular system(s) involved. For instance, the translation process1300 may be used to translate particular inodes into file pathnamesshorter than an absolute file pathname, such as for example a relativepathname.

In certain embodiments, the three-column database 1120 providessignificant advantages over a flat two-column table (e.g., with an inodecolumn and an absolute file pathname column). For instance, thethree-column database structure of the pathname database 1120 may useless memory than the two-column table and/or expedite folder renameoperations. As an example, when a name of a folder is modified, thethree-column database structure allows for a single lookup andmodification (e.g., modifying the short name column 1224 entryassociated with the entry inode column 1226 entry of the subject inode),while the two-column table would require multiple lookups andmodifications corresponding to each entry having an absolute filepathname that includes the folder to be renamed.

As discussed above, in certain embodiments, the pathname database 1120is maintained in userland (e.g., an application space external to thekernel space). In such embodiments, the pathname database 1120 may beadvantageously managed and/or accessed by userland code withoutimpacting the resources of the operating system kernel or otherapplications.

In certain embodiments, the pathname database 1120 may be initiallypopulated during an initialization period. For instance, a snapshot maybe taken to produce a static image of the file system of the sourcesystem. The pathname database 1120 may then be populated based on thesnapshot. As subsequent changes are made to file names of the sourcesystem, corresponding changes are made in the pathname database 1120 inorder to maintain synchronization.

In yet other embodiments, the pathname database 1120 may be specific tothe files and/or folders of one or more particular applications. Forexample, the pathname database 1120 may include inodes, short names andrelated information only for those inodes affected by a singleapplication (e.g., MICROSOFT EXCHANGE). In yet other embodiments,multiple pathname databases 1120 may be used.

As discussed above, certain embodiments of the invention allow one ormore users to customize the replication processes performed by one ormore replication systems described herein. FIGS. 14A and 14B illustrateblock diagrams of exemplary embodiments of user interface portions of areplication system.

In particular, FIG. 14A illustrates a user interface portion 1400 havingan interface 1402 associated with a client computer 1410. In certainembodiments, the interface 1402 comprises a graphical user interface.For instance, the graphical user interface may comprise a one or morewindows, drop down lists, text boxes, combinations of the same or thelike, displayed on a computer monitor.

The client computer 1410 further includes a filter driver 1412 thatcommunicates with a plurality of source logs 1420, 1422, 1424, 1426. Incertain embodiments, each of the source logs 1420, 1422, 1424, 1426corresponds to a particular application 1408.

In certain embodiments, the interface 1402 provides the user with anoption of specifying one or more policies that apply to each of theapplications 1408. For instance, the user may be capable of determiningwhich of the applications 1408 is to be monitored and/or how frequentlydata associated with a particular application is to be copied. In yetother embodiments, the interface 1402 may allow a user to associateparticular applications with particular source logs. For example, theuser may specify that all data files related to an SQL-relatedapplication be filtered and logged in source log 1420. The user may alsospecify that all data files related to a MICROSOFT EXCHANGE applicationis to be filtered and logged in source log 1422.

In yet other embodiments, the user, through the interface 1420, maydesignate different types of data management operations (e.g., storageoperations) to be performed on the application data by the filter driver1412. Storage operations may include, for example, creating snapshotcopies, application specific backup data, system related backup data,and/or other copy operations associated with data in a storage operationsystem.

In other embodiments, filter preferences may be set forth in a templateor default storage policy. For example, in the user interface portion1450 illustrated in FIG. 14B, a single source log 1420 may be used tolog files associated with different applications specified by the user.For example, within the source log 1420, the log entries may compriseone or more application identifier fields, such as for example, theapplication type field 508 (see FIG. 5), for distinguishing betweenlogged entries associated with different user selectable applications.

In certain embodiments, the interface 1402 is included within a sourcesystem, such as with the client computer 1410. In yet other embodiments,the interface 1402 may be part of a system management component, such asthe storage manager 680 of FIG. 6.

In certain embodiments of the invention, data replication systems andmethods may be used in a modular storage management system, embodimentsof which are described in more detail in U.S. Pat. No. 7,035,880, issuedApr. 5, 2006, which is hereby incorporated herein by reference in itsentirety. For example, the data replication system may be part of astorage operation cell that includes combinations of hardware andsoftware components directed to performing storage operations onelectronic data. Exemplary storage operation cells usable withembodiments of the invention include CommCells as embodied in the QNetstorage management system and the QiNetix storage management system byCommVault Systems, Inc. (Oceanport, N.J.), and as further described inU.S. patent application Ser. No. 10/877,831, filed Jun. 25, 2004, nowpublished as U.S. Patent Application Publication No. 2005-0033800 A1,which is hereby incorporated herein by reference in its entirety.

Systems and modules described herein may comprise software, firmware,hardware, or any combination(s) of software, firmware, or hardwaresuitable for the purposes described herein. Software and other modulesmay reside on servers, workstations, personal computers, computerizedtablets, PDAs, and other devices suitable for the purposes describedherein. Software and other modules may be accessible via local memory,via a network, via a browser, or via other means suitable for thepurposes described herein. Data structures described herein may comprisecomputer files, variables, programming arrays, programming structures,or any electronic information storage schemes or methods, or anycombinations thereof, suitable for the purposes described herein. Userinterface elements described herein may comprise elements from graphicaluser interfaces, command line interfaces, and other interfaces suitablefor the purposes described herein.

Embodiments of the invention are also described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products. It will be understood that eachblock of the flowchart illustrations and/or block diagrams, andcombinations of blocks in the flowchart illustrations and/or blockdiagrams, may be implemented by computer program instructions. Thesecomputer program instructions may be provided to a processor of ageneral purpose computer, special purpose computer, or otherprogrammable data processing apparatus to produce a machine, such thatthe instructions, which execute via the processor of the computer orother programmable data processing apparatus, create means forimplementing the acts specified in the flowchart and/or block diagramblock or blocks.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to operate in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the acts specified in the flowchart and/or block diagramblock or blocks. The computer program instructions may also be loadedonto a computer or other programmable data processing apparatus to causea series of operations to be performed on the computer or otherprogrammable apparatus to produce a computer implemented process suchthat the instructions which execute on the computer or otherprogrammable apparatus provide steps for implementing the acts specifiedin the flowchart and/or block diagram block or blocks.

In addition, methods and functions described herein are not limited toany particular sequence, and the acts or blocks relating thereto can beperformed in other sequences that are appropriate. For example,described acts or blocks may be performed in an order other than thatspecifically disclosed, or multiple acts or blocks may be combined in asingle act or block.

While certain embodiments of the inventions have been described, theseembodiments have been presented by way of example only, and are notintended to limit the scope of the disclosure. Indeed, the novel methodsand systems described herein may be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods and systems described herein may be made withoutdeparting from the spirit of the disclosure. The accompanying claims andtheir equivalents are intended to cover such forms or modifications aswould fall within the scope and spirit of the disclosure.

What is claimed is:
 1. A method for performing data managementoperations, the method comprising: monitoring operations associated witha computer application, the operations operative to write data to afirst location on a first storage device at a first time; storinginformation about the operations in a first log file on the firststorage device; copying at least a portion of the first log file on thefirst storage device to a second log file on a second storage device;processing the second log file to create replicated data at a secondlocation; generating a snapshot of the replicated data at a time afterthe second log file has been copied to the second storage device; andassociating the snapshot of the replicated data with the first locationwherein after the associating, the snapshot of the replicated dataappears to be a snapshot of the data existing at the first time on thefirst location.
 2. The method of claim 1, wherein monitoring operationscomprises processing the operations with a filter driver.
 3. The methodof claim 1 further comprising quiescing the computer application at atime of a known good state of the computer application.
 4. The method ofclaim 1, wherein processing the second log file comprises instantiatinga plurality of threads that process data entries in the second log fileand replay the data operations associated therewith to the secondlocation.
 5. The method of claim 4, wherein each of the plurality ofthreads process a particular type of data operation.
 6. The method ofclaim 1, wherein processing the second log file comprises replaying tothe second location device a first portion of data operations associatedwith data entries in the second log file.
 7. The method of claim 6,wherein processing the second log file comprises: replaying to a thirdstorage device data operations of a second portion of the data entriesto replicate data to a third location; performing a second storageoperation to copy replicated data from the third location to a fourthlocation; and associating the copy of the replicated data at the fourthlocation with a time of a known good state of the computer application.8. The method of claim 7, additionally comprising merging the copy ofthe replicated data at the second location with the copy of thereplicated data at the fourth location.
 9. The method of claim 1,wherein the second storage device further comprises a plurality ofvolumes.
 10. The method of claim 9, wherein processing the second logfile comprises at least two replication servers, each of the replicationservers being associated with a different one of the plurality ofvolumes.
 11. A system for performing data management operations, thesystem comprising: a first storage device that stores data writeoperations from at least one computer application at a first location ata first time; at least one module configured to monitor the data writeoperations and to generate first log entries based on the data writeoperations; a second storage device comprising second log entrieswherein the second log entries comprise copies of at least a portion ofthe first log entries; and a replication module in communication withthe second storage device and configured to: process the copied datawrite operations in the second log entries to create replicated datastored in a second location, associate the replicated data stored in thesecond location with the at least one computer application at the firstlocation, wherein after the associating, the replicated data appears tobe data existing at the first time on the first location.
 12. The systemof claim 11 further comprising a filter driver that is configured tomonitor operations associated with the computer application.
 13. Thesystem of claim 11 wherein the system is configured to quiesce thecomputer application at a time of a known good state of the computerapplication.
 14. The system of claim 11, wherein the replication moduleis configured to instantiate a plurality of threads that process dataentries in the second log file and replay the data operations associatedtherewith to the second location.
 15. The system of claim 14, whereineach of the plurality of threads process a particular type of dataoperation.
 16. The system of claim 11, wherein the replication module isconfigured to replay to the second location a first portion of dataoperations associated with data entries in the second log file.
 17. Thesystem of claim 16, wherein the replication module is configured to:replay to a third storage device data operations of a second portion ofthe data entries to replicate data to a third location; perform a secondstorage operation to copy replicated data from the third location to afourth location; and associate the copy of the replicated data at thefourth location with a time of a known good state of the computerapplication.
 18. The system of claim 17, additionally comprising mergingthe copy of the replicated data at the second location with the copy ofthe replicated data at the fourth location.
 19. The system of claim 11,wherein the second storage device further comprises a plurality ofvolumes.
 20. The system of claim 19, wherein processing the second logfile comprises at least two replication servers, each of the replicationservers being associated with a different one of the plurality ofvolumes.